Data Engineer 

About ConsenSys Data

ConsenSys Data sits within Consensys Software Inc. to help address all our variants of data, break down silos, enable best practices, provide first rate resources, and accelerate our mission of becoming a cutting edge data driven organization.  We are using a mix of providing some centralized data engineering functions as a shared service, while enabling our business units to make great data decisions with their own data functions.

ConsenSys Software Inc. is a wide organization with each individual business unit facing unique data challenges. Infura needs to provide real time analytics on top of a data pipeline doing billions of events per day. MetaMask Swaps needs to provide financial accounting for a purely on chain data source. Truffle needs to track developer engagement across the open source ecosystem.

We are looking for a data engineer to join our shared data engineering team, with a goal of helping to build, maintain and evolve our data warehouse to support the organization.

You will join a team to work with the business to ensure we have a first class data warehouse supporting our business units and our business decision making. Some of the key areas you will help ensure we are doing well are:

• data quality 
• data governance
• master data management

Responsibilities:

• Understanding of the business and data strategy
• Contribute to the collection, storage, management, quality, and protection of data
• Implementing data privacy policies and complying with data protection regulations
• Effectively communicate the status, value, and importance of data collection to executive members and staff
• Knowledge of relevant applications, big data solutions, and tools
• Knowledge of real time streaming data pipelines
• Governance: Advising on, monitoring, and governing enterprise data
• Operations: Enabling data usability, availability, and efficiency
• Innovation: Driving enterprise digital transformation innovation, cost reduction, and revenue generation
• Analytics: Supporting analytics and reporting on products, customers, operations, and markets

Who we’re looking for:

• 5+ years of work experience in an enterprise engineering domain
• Excellent problem-solving skills and sharp attention to detail
• Solid written and verbal communication skills
• Proficient in Python programming and well versed with popular frameworks like Pandas, Flask, Airflow, Apache Beam etc.
• Experience with any one cloud technologies i.e GCP, AWS or Azure (GCP preferred)
• Hands on experience with Cloud function, Big Query, DataFlow, Cloud Run, Airflow etc.
• Strong with Linux Commands and Shell Scripting
• Experience with Docker & Kubernetes
• Experience building CI/CD pipelines
• Knowledge of any one SCM tools like Git, BitBucket etc.
• Strong  knowledge on Terraform scripting
• Strong skills with writing SQL
• Enthusiasm for shipping high-quality code and helping peers do the same
• Proactiveness and be self-driven to be successful working in a remote environment
• Understanding of web development practices and terminology
• Relevant knowledge of cloud security and data security
• A belief in our mission and values

Bonus points:

• Blockchain expertise
• Hands-on experience with Kafka
• Hand-on experience with Apache Spark (py-spark preferred)
• You’re a MetaMask user!