Job summary
Seeking a Data Engineer with expertise in web scraping to design and maintain automated data collection systems and pipelines. The role involves close collaboration with analysts and engineers to build reliable data solutions supporting business decisions, offering autonomy in projects within a technology-driven environment in Norway. Experience with data processing, automation, and cloud technologies is valued.
Location and work setup
- Location
- Norway
- Work setup
- Remote
Responsibilities
Design, develop, and maintain scalable web scraping solutions and data pipelines to collect structured and unstructured data. Collaborate with stakeholders to understand data needs and deliver tailored datasets. Clean, transform, and validate large datasets using Python and Pandas. Manage data ingestion into databases or data warehouses. Use workflow orchestration tools like Apache Airflow to schedule and optimize data workflows. Ensure data quality and resolve production issues promptly. Improve internal tools and automation frameworks in collaboration with engineering teams.
Qualifications
Bachelor's or Master's degree in Computer Science or related field. 2 to 4 years of software development experience. Strong programming skills in Python, along with solid SQL and database knowledge. Advanced use of the Pandas library for data operations. Experience with web technologies including HTML, JavaScript, and APIs, plus familiarity with web scraping frameworks and tools such as Selenium, Scrapy, XPath, Fiddler, or Postman. Knowledge of workflow orchestration tools like Apache Airflow, containerization with Docker, and CI/CD pipelines (e.g., Jenkins or GitLab CI/CD). Preferably experience with AWS cloud services. Strong analytical and communication skills with a proactive approach to problem-solving and automation.