The suitable candidate will function as the lead in projects related to all aspects of collecting and preparing data for analytical or operational uses. The Data Engineer responsibilities include building data pipelines to pull together information from different source systems; integrating, consolidating and cleansing data; and structuring it for use in individual analytics applications as well as assisting with the overall data architecture. The candidate will work with domain experts, data scientists, analyst and engineers who are analyzing and visualizing the data to drive efficiencies and business decisions.
- Supporting and providing data in a ready-to-use form to data scientists who are looking to run queries and algorithms against the information for predictive analytics, machine learning and data mining purposes.
- The Data Engineer will have exposure to all business units, plants, processes, and products in order to work with data scientists, business analyst and other analytics community members.
- Responsible for compiling and installing database systems, writing complex queries, scaling to multiple machines, DB administration, and putting disaster recovery systems into place.
- The ability to build a robust, fault-tolerant data pipeline that cleans, transforms, and aggregates unorganized and messy data into databases or data sources.
- Work with business units and departments to deliver data aggregations to executives, business analysts and other end users for more basic types of analysis to aid in ongoing operations.
- Recommending new data streams and ensuring data integrity
- Identifying new projects and opportunities, and the associated data requirements
- Manage both structured and unstructured data sets — must be versed in different approaches to data architecture and applications including traditional Data warehousing concepts and Data Lakes.
- Skilled in extract, transform and load tools (Data Services, Informatica, or SSIS) and REST-oriented APIs for creating and managing data integration jobs, and providing data analysts and business users with simplified access to prepared data sets.
- Practical understanding of statistics, predictive analytics, machine learning, and artificial intelligence algorithms concepts which may include C#, Java, Python, Ruby, Scala and SQL.
- Database management and query languages
- Hadoop, MapReduce, Hive, Pig, MySQL, MongoDB, Cassandra, Spark, Data streaming, NoSQL, SQL, programming
- Ability to understand a variety of applications and domains (embedded sensors, plant processes, shop floor, ERP,etc.)
- Excellent written and verbal communication skils
- Must be able to work with all relevant technical and non-technical parties to ensure that quality results are obtained.
- B.S in Computer Science, Engineering or related disciplines with experience in data engineer and database administration.
- Willingness to travel as much as 30% of the time.
- Desire to work in a dynamic environment and have a continuous improvement and development mindset