As a Data Engineer, you will be responsible for extracting data from various media social platforms and APIs, developing and operating data pipelines, implementing MLOps practices, serving APIs, performing web scraping, and automating data processes. Your expertise in machine learning and proficiency in technology stacks such as GCP, Linux, BigQuery, and Airflow will be essential in delivering high-quality data solutions.

 

Key Accountabilities

  • Extract data from media social platforms (e.g., Facebook, Twitter, Instagram) and APIs using appropriate tools and techniques.
  • Develop, deploy, and maintain scalable and efficient data pipelines to ingest, transform, and load data from various sources into the data warehouse.
  • Collaborate with data scientists and analysts to understand data requirements and design optimal solutions.
  • Implement MLOps practices, including model deployment, monitoring, and version control, to ensure smooth integration of machine learning models into production environments.
  • Serve APIs to enable real-time access to data for internal and external stakeholders.
  • Perform web scraping to gather relevant data from websites and integrate it into the data ecosystem.
  • Automate data processes, ensuring efficiency, reliability, and accuracy in data handling and processing.
  • Monitor and optimize data pipelines and workflows for performance, reliability, and scalability.
  • Collaborate with cross-functional teams, including data scientists, analysts, and software engineers, to implement end-to-end data solutions.
  • Stay up-to-date with emerging technologies, industry trends, and best practices in data engineering, machine learning, and cloud platforms.

 

Skills/ Qualifications/ Experience

  • Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
  • 3+ years of professional experience as a Data Engineer or a similar role.
  • Proficiency in data extraction from media social platforms (e.g., Facebook Graph API, Twitter API, Instagram API) and other external data sources.
  • Strong experience in developing and operating data pipelines using tools like Apache Airflow or similar workflow management systems.
  • Solid understanding of MLOps principles and experience deploying machine learning models into production environments.
  • Knowledge of web scraping techniques and tools (e.g., Beautiful Soup, Scrapy).
  • Strong programming skills in languages such as Python, SQL, and familiarity with Linux environments.
  • Experience working with cloud platforms, especially Google Cloud Platform (GCP), including services like BigQuery, Dataflow, Pub/Sub, and Cloud Storage.
  • Ability to design, optimize, and tune data pipelines for performance, scalability, and reliability.
  • Excellent problem-solving skills and attention to detail.
  • Strong communication and collaboration skills to work effectively in a cross-functional team environment.

 

Preferred

  • Experience with cloud platforms, such as GCP, AWS, or Azure.
  • Knowledge of machine learning concepts and experience working with ML frameworks (e.g., TensorFlow, PyTorch).
  • Familiarity with containerization technologies (e.g., Docker, Kubernetes).

 

Our Offer

  • Work with a young and vibrant team in a fast-paced startup environment, embracing new and exciting challenges daily.
  • Opportunity to work on exciting projects, leveraging your skills in media social data extraction, data pipeline development, MLops, web scraping and automation.
  • Stay ahead of the curve by upskilling your tech knowledge and adapting to the latest technologies in the field.
  • Experience upgrading career development within the company, unlocking new and exciting opportunities.
  • Enjoy an attractive remuneration package that rewards your dedication and commitment.
  • Benefit from flexible working arrangements, including a hybrid mode setup
  • Engage in monthly team events/activities that foster camaraderie and create lasting memories.
  • Join a team that values work-life balance and supports you in finding the perfect equilibrium

 

If this opportunity captures your interest, we welcome you to share your CV/Resume along with your introduction, notice period (if applicable), and your expected salary range to najlaa.asyiqa@revmedia.my