Big Data Lead

Overview

DemandMatrix is on the lookout for a Big Data Lead with considerable experience and expertise in big data technologies to elevate the company's capabilities in solving complex data challenges. This role is essential for organizations looking to leverage cutting-edge big data tools to transform their data into actionable insights. The position will primarily focus on hands-on coding and implementation, making it imperative for candidates to have a solid grounding in software development and data engineering.

Responsibilities

The responsibilities of the Big Data Lead involve developing and managing large-scale text data processing, event-driven data pipelines, and performing in-memory computations. As part of this role, you will be:

• Working significantly on hands-on coding, suggesting an active involvement in engineering tasks.
• Optimizing various aspects of the data pipeline, considering elements such as CPU core performance, network I/O, and disk I/O.
• Utilizing cloud-native services within environments like Amazon Web Services (AWS) and Google Cloud Platform (GCP).

Required Skills

To be successful in this role, candidates need to meet various technical and professional criteria:

Must-Have Skills:

• 7+ years of hands-on experience in Software Development, specifically focusing on big data and large data processing pipelines.
• A minimum of 3 years of experience in building services and pipelines using Python.
• Expertise with various data processing systems, including both streaming and batch processing, with knowledge of tools such as Spark and Hadoop/MapReduce.
• Familiarity with at least one NoSQL store, such as MongoDB, Elasticsearch, or HBase.
• Understanding of data models, sharding, and data location strategies for distributed data environments, particularly in large-scale, high-throughput scenarios.
• Experience running scalable and highly available systems using either AWS or GCP.

Good-to-Have Skills:

• Familiarity with containerization tools like Docker and Kubernetes.
• Experience with CI/CD best practices.
• Knowledge in crawling or scraping techniques.

Work Environment

The position at DemandMatrix offers a flexible work environment. Every employee enjoys the benefit of remote work, allowing them to manage their schedules effectively. This model supports collaboration across the globe, enhancing productivity and job satisfaction, all while maintaining a work-life balance. Additionally, a Birthday Leave is offered, which adds to the company’s employee-centric policies.

Company Vision

At DemandMatrix, the overall vision is to disrupt the $100 billion sales and marketing intelligence industry through advanced machine learning and artificial intelligence solutions. The company prides itself on its innovative approaches to identifying potential customers for leading Fortune 100 companies such as Microsoft, Google, Adobe, Amazon, and IBM. The company's successful integration of technology in sales and marketing gives it a competitive edge, and being part of the DemandMatrix team means contributing to this exciting journey.

Candidate Experience

Ideal candidates for the Big Data Lead position at DemandMatrix will not only possess the required technical skills but will also have a genuine passion for technology and a drive to embark on project developments from scratch. It is essential that candidates have experience designing and building multiple big data modules, indicating an ability to handle complexity and innovate.

Conclusion

In summary, the Big Data Lead position at DemandMatrix is an exciting opportunity to work in a dynamic, technology-driven environment. This role is ideal for individuals who are experienced in big data technologies and passionate about leveraging data to drive business success. Those with the required skill set and experience will find this position rewarding, both in terms of personal development and contribution to the company's overall goals.

This job offer was originally published on himalayas.app