Data Engineer

We are looking for a Data Engineer to join our Engineering team to work with our data pipeline. We ingest data from some of the most interesting sources in the world, collecting from places like the Twitter and Foursquare APIs and even UFO sightings reported to the National UFO Reporting Center.

Leverage your expertise with data by contributing to our mission to make data more accessible for the rest of the world. Members of our Data Team make it possible for us to serve our customers with data like Trstrank and everything you can find in our Geo APIs. We are a world-class big data shop, with a unique approach and philosophy that you won’t be able to find anywhere else.

Our data pipeline uses technologies such as HBase, Elastic Search, Flume, Chef, Pig, and Hadoop. We’ve even developed tools of our own to make the ingestion pipeline run more smoothly, like a Ruby-based interface to Hadoop and a bulk loader for Elastic Search, that you can check out here: http://www.infochimps.com/labs

Experience with some of the following is preferred:

• HBase
• Hadoop
• Elastic Search
• Chef
• Java
• Pig
• Ruby

Other useful skills include:

• Natural Language Processing algorithms
• ETL (Extract, Transform, and Load) Experience
• Voronoi Diagrams
• Unsupervised clustering algorithms
• Large scale data processing

Ops Engineer

We are looking for an Operations Engineer to join our Engineering team to work with our big data architecture. If you have demonstrated ability to keep large numbers of systems running, balance shifting concerns from many stakeholders, and learn three new things a day, we want to talk to you. Leverage your expertise working with scalable architectures to help maintain, solidify, and scale our production environments and custom deployment stacks.

We are a world-class big data shop, with a unique approach and philosophy that you won’t be able to find anywhere else. We’re the authors of Cluster Chef, the premiere way to manage clusters in the cloud (https://github.com/infochimps/cluster_chef). Our data pipeline uses technologies such as HBase, Elastic Search, Flume, Pig, and Hadoop. We’ve also developed tools of our own to make the ingestion pipeline run more smoothly, like a Ruby-based interface to Hadoop and a bulk loader for Elastic Search, that you can check out at http://www.infochimps.com/labs.

Experience with some of the following is preferred:

• Chef
• Linux
• Amazon Web Services (or similar cloud IaaS providers)
• Flume
• HBase
• Hadoop
• Elastic Search
• Java
• Pig
• Ruby

Data Scientist

We are looking for a Data Scientist to work with our Engineering team to map the future of our big data infrastructure. For the past year our Engineering team has built up an amazing infrastructure for hosting and distributing the world’s data and now it’s time to take it to the next level. We utilize over half-a-dozen different best-in-class databases and tools including HBase, Elastic Search, Flume, Chef, Pig, and Hadoop. All these technologies work together to form a world-class platform for collecting and distributing data.

Core to our philosophy, and our primary mission, is the democratization of the world’s data. For some public examples of our projects, see our labs page at http://www.infochimps.com/labs . You are an ideal candidate if you enjoy working on big problems and having a big impact early in a company’s life.

This is an opportunity to work with a team where data science is core to the company’s mission, rather than a bolt-on to round out the engineering department. In addition to feeling culturally right at home, the Infochimps Data Scientist will have significant input into the team’s architectural approach and execution decisions. We’re looking for a hands-on coder who enjoys implementing algorithms as much as designing and tuning them.

Experience with some of the following is preferred:

• Pig
• HBase
• Hadoop
• Java
• Ruby

Other useful skills include:

• Natural Language Processing algorithms
• ETL (Extract, Transform, and Load) experience
• Voronoi diagrams
• Unsupervised clustering algorithms
• Supervised classification algorithms
• Large scale data processing experience

Architect

We are looking for an experienced Architect to work with our Engineering team to map the future of our big data infrastructure. For the past year our Engineering team has built up an amazing infrastructure for hosting and distributing the world’s data and now it’s time to take it to the next level. We utilize over half-a-dozen different best-in-class databases and tools including HBase, Elastic Search, Flume, Chef, Pig, and Hadoop. All these technologies work together to form a world-class platform for collecting and distributing data.

Core to our philosophy, and our primary mission, is the democratization of the world’s data. This backend infrastructure is critical to our product and progress towards this goal. Your contributions would help the rest of the world by taking the monkey-work out of dealing with data.

For some public examples of our projects, see our labs page at http://www.infochimps.com/labs .

You are an ideal candidate if you enjoy working on big problems and having a big impact early in a company’s life. You should have a deep understanding of design patterns, the Unix way, and a fingerspitzengefühl [intuitive feel] for maintaining infrastructure, untangling bugs, and simplifying systems.

Experience with some of the following is preferred:

• HBase
• Hadoop
• Elastic Search
• Chef
• Java
• Pig
• Ruby

Other useful skills include:

• Natural Language Processing algorithms
• ETL (Extract, Transform, and Load) Experience
• Voronoi Diagrams
• Unsupervised clustering algorithms
• Large scale data processing