Key Responsibilities:
- Assist in the development and maintenance of data pipelines using Databricks, Fabric, and PySpark.
- Support the integration and orchestration of data using Fabric tools.
- Write and optimize SQL queries for data extraction and manipulation.
- Collaborate with senior engineers and data scientists to understand and fulfill data requirements.
- Monitor and troubleshoot data processing issues to ensure smooth operations.
- Contribute to data governance and quality assurance processes.
- Keep up-to-date with the latest trends and best practices in data engineering.
Qualifications: - Bachelor’s degree in Computer Science, Engineering, or a related field.
- 2-3 years of experience in data engineering with hands-on experience in Databricks, Fabric, and
PySpark. - Proficiency in SQL for data querying and management.
- Basic understanding of cloud platforms (e.g., Azure, AWS, Google Cloud) and big data technologies.
- Familiarity with data warehousing concepts and ETL processes.
- Strong problem-solving skills and attention to detail.
- Ability to work both independently and collaboratively in a team environment.
- Good communication skills and a willingness to learn and grow in the field.
Preferred Skills: - Experience with additional data processing tools such as Apache Kafka or Apache Hive.
- Knowledge of data privacy and security best practices.
- Familiarity with Agile methodologies.