Job Summary
Experienced Big Data Engineer with strong expertise in Java, Apache Spark, and Google Cloud Platform (GCP). Adept at designing and developing scalable data pipelines, building high-performance microservices, and optimizing large-scale data processing systems. Proven ability to work with both batch and real-time data pipelines, leveraging modern cloud-native tools such as Dataflow, Dataproc, BigQuery, and Pub/Sub to deliver efficient and reliable data solutions.
Job Description
- Design and develop scalable batch and real-time data ingestion pipelines using Cloud Dataflow (Apache Beam) and Cloud Dataproc (Spark/Hadoop).
- Build and maintain robust, high-throughput RESTful microservices and backend applications using Core Java, Java 8, and Spring Boot.
- Develop and optimize BigQuery data warehouses, focusing on efficient storage design, partitioning, and query performance for large datasets.
- Implement event-driven streaming architectures using Cloud Pub/Sub to process high-velocity data with low latency.
- Migrate legacy Hadoop, MapReduce, and Hive workloads to modern GCP-based architectures, ensuring scalability and cost efficiency.
- Optimize Spark SQL and PySpark jobs through performance tuning, caching, partitioning, and resource management to handle terabyte-scale data.
- Collaborate with cross-functional teams to define data models, ETL/ELT processes, and data governance standards.
- Implement CI/CD pipelines and follow best practices for version control, testing, and deployment using tools like Jenkins/GitHub Actions.
- Ensure high availability, fault tolerance, and monitoring of data pipelines and applications.
Profile Description
- A highly skilled Big Data professional with a strong background in Java-based backend development and distributed data processing systems.
- Demonstrates deep expertise in Apache Spark, Hadoop ecosystem, and GCP services, with hands-on experience in building scalable, cloud-native data solutions. Proficient in multi-threading, concurrency, and Java collections, along with modern development practices using Spring Boot microservices architecture.
- Strong knowledge of data engineering concepts, including data modeling, ETL/ELT pipelines, and real-time data processing.
- Experienced in leveraging BigQuery, Dataflow, Dataproc, and Pub/Sub to design efficient data pipelines and analytics platforms.
- Capable of optimizing performance for large-scale systems while balancing cost and resource utilization.
- A proactive problem-solver with excellent analytical skills, capable of working in fast-paced environments and delivering high-quality, scalable solutions aligned with business requirements.
About Company
A client of ilink Talent Solutions is a global digital engineering and data analytics firm founded in 1991, headquartered in the U.S., with a workforce of 3,500+ professionals and delivery centers across North America, India, Canada, Australia, and beyond. They specialize in cloud engineering, data platforms, AI/GenAI, and enterprise analytics, helping Fortune 1000 companies accelerate digital transformation. Known for proprietary accelerators like LeapLogic™ and strong partnerships with AWS, Databricks, and Snowflake, the company is recognized for driving innovation, large-scale modernization, and high-impact business outcomes.
