Jobless Developer
Brillio logo

Brillio

Posted 20 days ago

Lead Data Engineer

ChennaiHybridFull-time

AI Summary

Lead data engineering role focused on building cloud-native data pipelines and AI-ready data platforms on Azure for the AI & Data Engineering practice.

About this role

Primary Skills

  • AKS, Event Hub, Azure DevOps, Cosmos DB, Azure Functions

Specialization

  • Azure Data Engineering Advanced: Senior Data Engineer

Job requirements

Job Description

We are seeking a Senior Data Engineer with strong hands‑on expertise in ** Azure Data Platform and Microsoft Fabric** to design, build, and scale modern, enterprise‑grade data solutions. The role involves working on end‑to‑end data engineering pipelines, lakehouse architectures, and analytics‑ready data models using Azure and Fabric’s unified analytics capabilities.


Key Responsibilities

*   Design, develop, and manage **end‑to‑end data pipelines** using ** Microsoft Fabric (Data Pipelines, Lakehouse, Warehouse, OneLake)** and ** Azure Data Factory**
*   Build scalable data transformation and processing solutions using **Fabric Notebooks, PySpark, Python, and SQL**
*   Implement **Lakehouse and Medallion Architecture (Bronze, Silver, Gold)** to support analytics and reporting workloads
*   Develop and optimize **SQL-based solutions** including ** T‑SQL, stored procedures, and performance‑tuned queries**
*   Integrate data from multiple sources including **RDBMS, APIs, files, streaming sources **, and cloud storage
*   Work with **ADLS Gen2, Blob Storage, and OneLake** for structured and unstructured data management
*   Enable analytics and reporting by integrating Fabric datasets with **Power BI**
*   Implement data quality checks, monitoring, logging, and error‑handling mechanisms
*   Support migration and modernization initiatives from **ADF / Databricks to Microsoft Fabric **, where applicable
*   Ensure adherence to **data governance, security, lineage, and compliance** standards (Microsoft Purview exposure preferred)
*   Collaborate closely with architects, analysts, and business stakeholders to translate requirements into scalable data solutions
*   Provide technical guidance and mentoring to junior engineers

Required Skills & Expertise

Microsoft Azure & Fabric

*   Microsoft Fabric (Data Pipelines, Lakehouse, Warehouse, OneLake)
*   Azure Data Factory (ADF)
*   Azure Data Lake Storage Gen2 (ADLS G2)
*   Azure Blob Storage

Programming & Query Languages

*   SQL (Basic & Advanced)
*   T‑SQL
*   Stored Procedures
*   Python
*   PySpark / Spark SQL

Data Engineering & Architecture

*   ETL / ELT Fundamentals
*   Modern Data Platform Concepts
*   Data Warehousing & Lakehouse Architecture
*   Data Modeling Fundamentals (Star / Snowflake schemas)
*   Incremental loads, CDC, and watermarking techniques

Monitoring & Governance

*   Pipeline monitoring & performance optimization
*   Azure Monitoring / Fabric monitoring tools
*   Microsoft Purview (Data lineage & governance – good to have)

Good to Have

*   Experience with **streaming or near real‑time processing**
*   Exposure to **Databricks** in hybrid Azure–Fabric environments
*   CI/CD for data pipelines using **Git / Azure DevOps**
*   Experience supporting enterprise‑scale analytics platforms

Soft Skills

*   Strong problem‑solving and analytical mindset
*   Excellent communication and stakeholder management skills
*   Ability to work independently in fast‑paced delivery environments
*   Ownership mindset with focus on quality and reliability

Why Join Us

*   Work on **cutting‑edge Azure & Microsoft Fabric implementations**
*   Exposure to large‑scale enterprise modernization programs
*   Growth opportunities across cloud, data platforms, and analytics
*   Collaborative, high‑impact engineering culture

We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.

Skills

AzureAKSAzure DevOpsAzure FunctionsCosmosDBMS FabricEvent Hub