Post job

Data Engineer jobs at ZP Group

- 20623 jobs
  • Senior Software Engineer (Python) - Remote

    ZP Group 4.0company rating

    Data engineer job at ZP Group

    Zachary Piper Solutions is seeking a Senior Software Engineer to support a high-impact federal cybersecurity initiative. This is a fully remote opportunity to contribute to the development of a large-scale cyber data visualization and analytics platform. Responsibilities: * Design, develop, and maintain backend Python applications for enterprise-level cybersecurity solutions. * Collaborate within an Agile (SAFe) team, including pair programming, to implement scalable and reliable data processing pipelines. * Develop and integrate ETL processes and RESTful APIs to support data ingestion, transformation, and analytics. * Work with persistent data storage systems such as RDS, DynamoDB, and S3, including experience with pushing/pulling data to Elasticsearch. * Utilize containerization technologies such as Docker and Kubernetes to deploy and manage microservice-based applications. * Write unit tests and implement event-driven and scheduled features to ensure code reliability and maintainability. * Participate in continuous improvement efforts and adapt to evolving cybersecurity requirements and technologies. * Contribute to system design, architecture discussions, and backend-focused full-stack development. Required Experience: * 10+ years of experience in software engineering, with a strong focus on backend Python development. * Proven experience building scalable, high-performance systems and ETL pipelines. * 5+ years working with data storage solutions (e.g., RDS, DynamoDB, S3). * 5+ years using container technologies (Docker, Kubernetes). * 5+ years using Git-based version control systems (GitHub, GitLab, Bitbucket). * Strong problem-solving and analytical skills with the ability to work independently. * Experience writing unit tests and implementing event-driven features. * Excellent communication skills and ability to collaborate in distributed teams. Preferred Qualifications: * Experience working in CI/CD environments and familiarity with infrastructure as code (IaC) tools. * 3+ years of experience in Agile development teams. * 5+ years working with AWS cloud services (flexibility for other cloud platforms). * Experience with SQS, Kafka, and Kibana for log processing and visualization. * Background in cybersecurity tools, technologies, and mission support. * AWS Certification is a plus. * Experience with leadership or mentorship in technical teams is a bonus. Compensation: Salary Range: $120,000 - $ 150,000 (depending on experience) Comprehensive Benefits: Medical, Dental, Vision, 401K, PTO, Sick leave as required by law, and Holidays Location: Fully remote work environment in EST Clearance: Ability to obtain and maintain a US government DSH Public Trust Security Clearance Keywords: Python, ETL, AWS, Docker, Kubernetes, Git, CI/CD, Agile, Cybersecurity, DHS, Kafka, Kibana, Infrastructure as Code, Data Pipelines
    $120k-150k yearly 4d ago
  • Senior CNO Developer

    Mantech 4.5company rating

    Annapolis, MD jobs

    MANTECH seeks a motivated, career and customer-oriented Senior CNO Developer to join our team in Annapolis Junction, Maryland. We're looking for a Senior Capability Developer to join our elite team. In this role, you'll apply your deep technical expertise to analyze, reverse-engineer, and develop mission-critical capabilities that directly support national security objectives. You will be a key player in a fast-paced environment, tackling unique challenges at the intersection of hardware, software, and embedded systems. Responsibilities include but are not limited to: Develop custom software tools and applications using Python, C, and Assembly, focusing on embedded and resource-constrained systems. Conduct rigorous code reviews to ensure the quality, security, and performance of developed software. Reverse engineer complex hardware and software systems to understand their inner workings and identify potential vulnerabilities. Perform in-depth vulnerability research to discover and analyze weaknesses in a variety of targets. Collaborate with a team of skilled engineers to design and implement innovative solutions to challenging technical problems. Minimum Qualifications: Bachelor's degree and 12 years of experience; or, a high school diploma with 16 years of experience; or, an Associate's degree with 14 years of experience. A Master's degree may substitute for 2 years of experience, and a PhD may substitute for 4 years of experience. Must have 7 years of position-relevant work experience Proficiency in programming and application development. Strong scripting skills, particularly in Python, C, and Assembly. Deep expertise in managing, configuring, and troubleshooting Linux. Experience in embedded systems. Experience in reverse engineering and vulnerability research of hardware and software. Experience in code review. Preferred Qualifications: Experience in CNO (Computer Network Operations) Development. Experience in virtualization. Knowledge of IoT (Internet of Things) devices. Experience with Linux Kernel development and sockets. Knowledge of integrating security tools into the CI/CD (Continuous Integration/Continuous Delivery) pipeline. Networking skills. Clearance Requirements: Must have a current/active Top Secret/SCI clearance. Physical Requirements: The person in this position must be able to remain in a stationary position 50% of the time. Occasionally move about inside the office to access file cabinets, office machinery, or to communicate with co-workers, management, and customers, via email, phone, and or virtual communication, which may involve delivering presentations
    $85k-109k yearly est. 3d ago
  • Snowflake Data Engineer (DBT SQL)

    Maganti It Resources, LLC 3.9company rating

    San Jose, CA jobs

    Job Description - Snowflake Data Engineer (DBT SQL) Duration: 6 months Key Responsibilities • Design, develop, and optimize data pipelines using Snowflake and DBT SQL. • Implement and manage data warehousing concepts, metadata management, and data modeling. • Work with data lakes, multi-dimensional models, and data dictionaries. • Utilize Snowflake features such as Time Travel and Zero-Copy Cloning. • Perform query performance tuning and cost optimization in cloud environments. • Administer Snowflake architecture, warehousing, and processing. • Develop and maintain PL/SQL Snowflake solutions. • Apply design patterns for scalable and maintainable data solutions. • Collaborate with cross-functional teams and tech leads across multiple tracks. • Provide technical and functional guidance to team members. Required Skills & Experience • Hands-on Snowflake development experience (mandatory). • Strong proficiency in SQL and DBT SQL. • Knowledge of data warehousing concepts, metadata management, and data modeling. • Experience with data lakes, multi-dimensional models, and data dictionaries. • Expertise in Snowflake features (Time Travel, Zero-Copy Cloning). • Strong background in query optimization and cost management. • Familiarity with Snowflake administration and pipeline development. • Knowledge of PL/SQL and SQL databases (additional plus). • Excellent communication, leadership, and organizational skills. • Strong team player with a positive attitude.
    $119k-167k yearly est. 2d ago
  • Data Engineer

    DL Software Inc. 3.3company rating

    New York, NY jobs

    DL Software produces Godel, a financial information and trading terminal. Role Description This is a full-time, on-site role based in New York, NY, for a Data Engineer. The Data Engineer will design, build, and maintain scalable data systems and pipelines. Responsibilities include data modeling, developing and managing ETL workflows, optimizing data storage solutions, and supporting data warehousing initiatives. The role also involves collaborating with cross-functional teams to improve data accessibility and analytics capabilities. Qualifications Strong proficiency in Data Engineering and Data Modeling Mandatory: strong experience in global financial instruments including equities, fixed income, options and exotic asset classes Strong Python background Expertise in Extract, Transform, Load (ETL) processes and tools Experience in designing, managing, and optimizing Data Warehousing solutions
    $91k-123k yearly est. 2d ago
  • Senior Data Engineer

    Cyber Space Technologies LLC 4.4company rating

    Boston, MA jobs

    Data Engineer (HRIS experience) Boston, MA ( 4 days onsite a week ) Key Responsibilities: Translate business needs into data modelling strategies and implement Snowflake data models to support HR analytics, KPIs, and reporting. Design, build, and maintain Snowflake objects including tables, views, and stored procedures. Develop and execute SQL or Python transformations, data mappings, cleansing, validation, and conversion processes. Establish and enforce data governance standards to ensure consistency, quality, and completeness of data assets. Manage technical metadata and documentation for data warehousing and migration efforts. Optimize performance of data transformation pipelines and monitor integration performance. Design, configure, and optimize integrations between Workday and third-party applications. Participate in system testing including unit, integration, and regression phases. Support data analysis needs throughout the implementation lifecycle. Required Experience & Skills: Experience with Snowflake or similar data warehouse platforms Strong SQL skills and experience with data transformation tools Experience with ETL processes and data validation techniques Understanding of HR data structures and relationships Excellent analytical and problem-solving abilities Experience with developing with Python Architecting a data warehousing solution leveraging data from Workday or other HRIS such as Workday to support advanced reporting and insights for an organization Preferred Experience & Skills: Experience in developing and supporting a data warehouse serving the HR domain Experience with data platforms where SCD Type 2 was required Experience with data visualization tools such as Tableau Experience with architecting or working with ELT technologies (such as DBT) and data architectures Understanding of HR processes, compliance requirements, and industry best practices
    $79k-109k yearly est. 2d ago
  • Azure Data Engineer

    Programmers.Io 3.8company rating

    Weehawken, NJ jobs

    · Expert level skills writing and optimizing complex SQL · Experience with complex data modelling, ETL design, and using large databases in a business environment · Experience with building data pipelines and applications to stream and process datasets at low latencies · Fluent with Big Data technologies like Spark, Kafka and Hive · Expert level understanding on Azure Data Factory, Azure Synapse, Azure SQL, Azure Data Lake, and Azure App Service is required · Designing and building of data pipelines using API ingestion and Streaming ingestion methods · Knowledge of Dev-Ops processes (including CI/CD) and Infrastructure as code is essential · Experience in developing NO SQL solutions using Azure Cosmos DB is essential · Thorough understanding of Azure and AWS Cloud Infrastructure offerings · Working knowledge of Python is desirable · Designing and implementing scalable and secure data processing pipelines using Azure Data Factory, Azure Databricks, and other Azure services · Managing and optimizing data storage using Azure Data Lake Storage, Azure SQL Data Warehouse, and Azure Cosmos DB · Monitoring and troubleshooting data-related issues within the Azure environment to maintain high availability and performance · Implementing data security measures, including encryption, access controls, and auditing, to protect sensitive information · Automating data pipelines and workflows to streamline data ingestion, processing, and distribution tasks · Utilizing Azure's analytics services, such as Azure Synapse Analytics, to provide insights and support data-driven decision-making. · Documenting data procedures, systems, and architectures to maintain clarity and ensure compliance with regulatory standards · Providing guidance and support for data governance, including metadata management, data lineage, and data cataloging Best Regards, Dipendra Gupta Technical Recruiter *****************************
    $92k-132k yearly est. 1d ago
  • Sr Data Platform Engineer

    The Judge Group 4.7company rating

    Elk Grove, CA jobs

    Hybrid role 3X a week in office in Elk Grove, CA; no remote capabilities This is a direct hire opportunity. We're seeking a seasoned Senior Data Platform Engineer to design, build, and optimize scalable data solutions that power analytics, reporting, and AI/ML initiatives. This full‑time role is hands‑on, working with architects, analysts, and business stakeholders to ensure data systems are reliable, secure, and high‑performing. Responsibilites: Build and maintain robust data pipelines (structured, semi‑structured, unstructured). Implement ETL workflows with Spark, Delta Lake, and cloud‑native tools. Support big data platforms (Databricks, Snowflake, GCP) in production. Troubleshoot and optimize SQL queries, Spark jobs, and workloads. Ensure governance, security, and compliance across data systems. Integrate workflows into CI/CD pipelines with Git, Jenkins, Terraform. Collaborate cross‑functionally to translate business needs into technical solutions. Qualifications: 7+ years in data engineering with production pipeline experience. Expertise in Spark ecosystem, Databricks, Snowflake, GCP. Strong skills in PySpark, Python, SQL. Experience with RAG systems, semantic search, and LLM integration. Familiarity with Kafka, Pub/Sub, vector databases. Proven ability to optimize ETL jobs and troubleshoot production issues. Agile team experience and excellent communication skills. Certifications in Databricks, Snowflake, GCP, or Azure. Exposure to Airflow, BI tools (Power BI, Looker Studio).
    $108k-153k yearly est. 4d ago
  • Senior Data Governance Consultant (Informatica)

    Paradigm Technology 4.2company rating

    Plano, TX jobs

    Senior Data Governance Consultant (Informatica) About Paradigm - Intelligence Amplified Paradigm is a strategic consulting firm that turns vision into tangible results. For over 30 years, we've helped Fortune 500 and high-growth organizations accelerate business outcomes across data, cloud, and AI. From strategy through execution, we empower clients to make smarter decisions, move faster, and maximize return on their technology investments. What sets us apart isn't just what we do, it's how we do it. Driven by a clear mission and values rooted in integrity, excellence, and collaboration, we deliver work that creates lasting impact. At Paradigm, your ideas are heard, your growth is prioritized, your contributions make a difference. Summary: We are seeking a Senior Data Governance Consultant to lead and enhance data governance capabilities across a financial services organization The Senior Data Governance Consultant will collaborate closely with business, risk, compliance, technology, and data management teams to define data standards, strengthen data controls, and drive a culture of data accountability and stewardship The ideal candidate will have deep experience in developing and implementing data governance frameworks, data policies, and control mechanisms that ensure compliance, consistency, and trust in enterprise data assets Hands-on experience with Informatica, including Master Data Management (MDM) or Informatica Data Management Cloud (IDMC), is preferred This position is Remote, with occasional travel to Plano, TX Responsibilities: Data Governance Frameworks: Design, implement, and enhance data governance frameworks aligned with regulatory expectations (e.g., BCBS 239, GDPR, CCPA, DORA) and internal control standards Policy & Standards Development: Develop, maintain, and operationalize data policies, standards, and procedures that govern data quality, metadata management, data lineage, and data ownership Control Design & Implementation: Define and embed data control frameworks across data lifecycle processes to ensure data integrity, accuracy, completeness, and timeliness Risk & Compliance Alignment: Work with risk and compliance teams to identify data-related risks and ensure appropriate mitigation and monitoring controls are in place Stakeholder Engagement: Partner with data owners, stewards, and business leaders to promote governance practices and drive adoption of governance tools and processes Data Quality Management: Define and monitor data quality metrics and KPIs, establishing escalation and remediation procedures for data quality issues Metadata & Lineage: Support metadata and data lineage initiatives to increase transparency and enable traceability across systems and processes Reporting & Governance Committees: Prepare materials and reporting for data governance forums, risk committees, and senior management updates Change Management & Training: Develop communication and training materials to embed governance culture and ensure consistent understanding across the organization Required Qualifications: 7+ years of experience in data governance, data management, or data risk roles within financial services (banking, insurance, or asset management preferred) Strong knowledge of data policy development, data standards, and control frameworks Proven experience aligning data governance initiatives with regulatory and compliance requirements Familiarity with Informatica data governance and metadata tools Excellent communication skills with the ability to influence senior stakeholders and translate technical concepts into business language Deep understanding of data management principles (DAMA-DMBOK, DCAM, or equivalent frameworks) Bachelor's or Master's Degree in Information Management, Data Science, Computer Science, Business, or related field Preferred Qualifications: Hands-on experience with Informatica, including Master Data Management (MDM) or Informatica Data Management Cloud (IDMC), is preferred Experience with data risk management or data control testing Knowledge of financial regulatory frameworks (e.g., Basel, MiFID II, Solvency II, BCBS 239) Certifications, such as Informatica, CDMP, or DCAM Background in consulting or large-scale data transformation programs Key Competencies: Strategic and analytical thinking Strong governance and control mindset Excellent stakeholder and relationship management Ability to drive organizational change and embed governance culture Attention to detail with a pragmatic approach Why Join Paradigm At Paradigm, integrity drives innovation. You'll collaborate with curious, dedicated teammates, solving complex problems and unlocking immense data value for leading organizations. If you seek a place where your voice is heard, growth is supported, and your work creates lasting business value, you belong at Paradigm. Learn more at ******************** Policy Disclosure: Paradigm maintains a strict drug-free workplace policy. All offers of employment are contingent upon successfully passing a standard 5-panel drug screen. Please note that a positive test result for any prohibited substance, including marijuana, will result in disqualification from employment, regardless of state laws permitting its use. This policy applies consistently across all positions and locations.
    $76k-107k yearly est. 3d ago
  • Data Engineer (AWS Redshift, BI, Python, ETL)

    Prosum 4.4company rating

    Manhattan Beach, CA jobs

    We are seeking a skilled Data Engineer with strong experience in business intelligence (BI) and data warehouse development to join our team. In this role, you will design, build, and optimize data pipelines and warehouse architectures that support analytics, reporting, and data-driven decision-making. You will work closely with analysts, data scientists, and business stakeholders to ensure reliable, scalable, and high-quality data solutions. Responsibilities: Develop and maintain ETL/ELT pipelines for ingesting, transforming, and delivering data. Design and enhance data warehouse models (star/snowflake schemas) and BI datasets. Optimize data workflows for performance, scalability, and reliability. Collaborate with BI teams to support dashboards, reporting, and analytics needs. Ensure data quality, governance, and documentation across all solutions. Qualifications: Proven experience with data engineering tools (SQL, Python, ETL frameworks). Strong understanding of BI concepts, reporting tools, and dimensional modeling. Hands-on experience with cloud data platforms (e.g., AWS, Azure, GCP) is a plus. Excellent problem-solving skills and ability to work in a cross-functional environment.
    $99k-139k yearly est. 1d ago
  • Senior Data Engineer

    Programmers.Io 3.8company rating

    Chicago, IL jobs

    requires visa independent candidates. Note: (OPT, CPT, H1B holders will not work at this time) Design, develop, and maintain scalable ETL pipelines using AWSGlue Collaborate with data engineers and analysts to understand data requirements Build and manage data extraction, transformation, and loading processes Optimize and troubleshoot existing Glue jobs and workflows Ensure data quality, integrity, and security throughout the ETL process Integrate AWS Glue with other AWS services like S3, Lambda, Redshift, and Step Functions Maintain documentation of data workflows and processes Stay updated with the latest AWS tools and best practices Required Skills Strong hands-on experience with AWS Glue, PySpark, and Python Proficiency in SQL and working with structured/unstructured data (JSON, CSV, Parquet) Experience with data warehousing concepts and tools Familiarity with CI/CD pipelines, Terraform, and scripting (PowerShell, Bash) Solid understanding of data modeling, data integration, and data management Exposure to AWS Batch, Step Functions, and Data Catalogs
    $81k-112k yearly est. 3d ago
  • Data Engineer

    The Judge Group 4.7company rating

    Jersey City, NJ jobs

    ONLY LOCALS TO NJ/NY - NO RELOCATION CANDIDATES Skillset: Data Engineer Must Haves: Python, PySpark, AWS - ECS, Glue, Lambda, S3 Nice to Haves: Java, Spark, React Js Interview Process: Interview Process: 2 rounds, 2nd will be on site You're ready to gain the skills and experience needed to grow within your role and advance your career - and we have the perfect software engineering opportunity for you. As a Data Engineer III - Python / Spark / Data Lake at JPMorgan Chase within the Consumer and Community Bank , you will be a seasoned member of an agile team, tasked with designing and delivering reliable data collection, storage, access, and analytics solutions that are secure, stable, and scalable. Your responsibilities will include developing, testing, and maintaining essential data pipelines and architectures across diverse technical areas, supporting various business functions to achieve the firm's business objectives. Job responsibilities: • Supports review of controls to ensure sufficient protection of enterprise data. • Advises and makes custom configuration changes in one to two tools to generate a product at the business or customer request. • Updates logical or physical data models based on new use cases. • Frequently uses SQL and understands NoSQL databases and their niche in the marketplace. • Adds to team culture of diversity, opportunity, inclusion, and respect. • Develop enterprise data models, Design/ develop/ maintain large-scale data processing pipelines (and infrastructure), Lead code reviews and provide mentoring thru the process, Drive data quality, Ensure data accessibility (to analysts and data scientists), Ensure compliance with data governance requirements, and Ensure business alignment (ensure data engineering practices align with business goals). • Supports review of controls to ensure sufficient protection of enterprise data Required qualifications, capabilities, and skills • Formal training or certification on data engineering concepts and 2+ years applied experience • Experience across the data lifecycle, advanced experience with SQL (e.g., joins and aggregations), and working understanding of NoSQL databases • Experience with statistical data analysis and ability to determine appropriate tools and data patterns to perform analysis • Extensive experience in AWS, design, implementation, and maintenance of data pipelines using Python and PySpark. • Proficient in Python and PySpark, able to write and execute complex queries to perform curation and build views required by end users (single and multi-dimensional). • Proven experience in performance and tuning to ensure jobs are running at optimal levels and no performance bottleneck. • Advanced proficiency in leveraging Gen AI models from Anthropic (or OpenAI, or Google) using APIs/SDKs • Advanced proficiency in cloud data lakehouse platform such as AWS data lake services, Databricks or Hadoop, relational data store such as Postgres, Oracle or similar, and at least one NOSQL data store such as Cassandra, Dynamo, MongoDB or similar • Advanced proficiency in Cloud Data Warehouse Snowflake, AWS Redshift • Advanced proficiency in at least one scheduling/orchestration tool such as Airflow, AWS Step Functions or similar • Proficiency in Unix scripting, data structures, data serialization formats such as JSON, AVRO, Protobuf, or similar, big-data storage formats such as Parquet, Iceberg, or similar, data processing methodologies such as batch, micro-batching, or stream, one or more data modelling techniques such as Dimensional, Data Vault, Kimball, Inmon, etc., Agile methodology, TDD or BDD and CI/CD tools. Preferred qualifications, capabilities, and skills • Knowledge of data governance and security best practices. • Experience in carrying out data analysis to support business insights. • Strong Python and Spark
    $79k-111k yearly est. 2d ago
  • Data Engineer AI Systems

    Mak Technologies LLC 3.7company rating

    Saint Louis, MO jobs

    Job Title: Data Engineer - AI Systems 6 Months St. Louis, MO Day 1 onsite role Data Engineer - AI Systems (Databricks) Primary Skills: Data Engineer, Databricks, Python, PySpark, AI/ML We'rebuilding intelligent, Databricks-powered AI systems that structure and activate information from diverse enterprise sources (Confluence, OneDrive, PDFs, andmore). As a Data Engineer, you'll design and optimize the data pipelinesthat transform raw and unstructured content into clean, AI-ready datasets formachine learning and generative AI agents. You'llcollaborate with a cross-functional team of Machine Learning Engineers,Software Developers, and domain experts to create high-quality data foundationsthat power Databricks-native AI agents and retrieval systems. KeyResponsibilities Develop Scalable Pipelines: Design, build, and maintain high-performance ETL and ELT workflows using Databricks, PySpark, and Delta Lake. Data Integration: Build APIs and connectors to ingest data from collaboration platforms such as Confluence, OneDrive, and other enterprise systems. Unstructured Data Handling: Implement extraction and transformation pipelines for text, PDFs, and scanned documents using Databricks OCR and related tools. Data Modeling: Design Delta Lake and Unity Catalog data models for both structured and vectorized (embedding-based) data stores. Data Quality & Observability: Apply validation, version control, and quality checks to ensure pipeline reliability and data accuracy. Collaboration: Work closely with ML Engineers to prepare datasets for LLM fine-tuning and vector database creation, and with Software Engineers to deliver end-to-end data services. Performance & Automation: Optimize workflows for scale and automation, leveraging Databricks Jobs, Workflows, and CI/CD best practices. What YouBring Experience with data engineering, ETL development, or data pipeline automation. Proficiency in Python, SQL, and PySpark. Hands-on experience with Databricks, Spark, and Delta Lake. Familiarity with data APIs, JSON, and unstructured data processing (OCR, text extraction). Understanding of data versioning, schema evolution, and data lineage concepts. Interest in AI/ML data pipelines, vector databases, and intelligent data systems. BonusSkills Experience with vector databases (e.g., Pinecone, Chroma, FAISS) or Databricks' Vector Search. Exposure to LLM-based architectures, LangChain, or Databricks Mosaic AI. Knowledge of data governance frameworks, Unity Catalog, or access control best practices. Familiarity with REST API development or data synchronization services (e.g., Airbyte, Fivetran, custom connectors
    $74k-102k yearly est. 1d ago
  • Data Engineer

    A2C 4.7company rating

    Alpharetta, GA jobs

    5 days onsite in Alpharetta, GA Skills required: Python Data Pipeline Data Analysis Data Modeling Must have solid Cloud experience AI/ML Strong problem-solving skills Strong Communication skill A problem solver with ability to analyze and research complex issues and problems; and proposing actionable solutions and/or strategies. Solid understanding and hands on experience with major cloud platforms. Experience in designing and implementing data pipelines. Must have experience with one of the following: GCP, AWS OR Azure - MUST have the drive to learn GCP.
    $77k-106k yearly est. 2d ago
  • Data Engineer w/ Python & SQL

    A2C 4.7company rating

    Alpharetta, GA jobs

    We're looking for a Data Engineer to build and maintain scalable data pipelines and cloud data infrastructure on GCP. The role focuses on BigQuery, Dataflow, and modern ETL/ELT to support analytics and ML workflows. MUST HAVES A problem solver with ability to analyze and research complex issues and problems; and proposing actionable solutions and/or strategies. Solid understanding and hands on experience with major cloud platforms. Experience in designing and implementing data pipelines. Must have strong Python, SQL & GCP skills Responsibilities Build and optimize batch/streaming pipelines using Dataflow, Pub/Sub, Composer. Develop and tune BigQuery models, queries, and ingestion processes. Implement IaC (Terraform), CI/CD, monitoring, and data quality checks. Ensure data governance, security, and reliable pipeline operations. Collaborate with data science teams and support Vertex AI-based ML workflows. Must-Have Must have strong Python, SQL & GCP skills 3-5+ years of data engineering experience. Hands-on GCP experience (BigQuery, Dataflow, Pub/Sub). Solid ETL/ELT and data modeling experience. Nice-to-Have GCP certifications, Spark, Kafka, Airflow, dbt/Dataform, Docker/K8s.
    $77k-106k yearly est. 3d ago
  • Data Engineer

    Lancesoft, Inc. 4.5company rating

    Austin, TX jobs

    We are seeking a Data Engineer to join a dynamic Agile team and support the build and enhancement of a large-scale data integration hub. This role requires hands-on experience in data acquisition, ETL automation, SQL development, and performance analytics. What You'll Do ✔ Lead technical work within Agile development teams ✔ Automate ETL processes using Informatica Power Center / IICS ✔ Develop complex Oracle/Snowflake SQL scripts & views ✔ Integrate data from multiple sources (Oracle, SQL Server, Excel, Access, PDF) ✔ Support CI/CD and deployment processes ✔ Produce technical documentation, diagrams & mockups ✔ Collaborate with architects, engineers & business stakeholders ✔ Participate in Sprint ceremonies & requirements sessions ✔ Ensure data quality, validation & accuracy Must Have Experience ✅ 8+ years: Informatica Power Center / IICS ETL workflow development SQL development (Oracle/Snowflake) Data warehousing & analytics Technical documentation (Visio/Erwin, MS Office, MS Project)
    $77k-102k yearly est. 2d ago
  • Azure Data Engineer Sr

    Resolve Tech Solutions 4.4company rating

    Irving, TX jobs

    Minimum 7 years of relevant work experience in data engineering, with at least 2 years in a data modeling. Strong technical foundation in Python, SQL, and experience with cloud platforms (Azure,). Deep understanding of data engineering fundamentals, including database architecture and design, Extract, transform and load (ETL) processes, data lakes, data warehousing, and both batch and streaming technologies. Experience with data orchestration tools (e.g., Airflow), data processing frameworks (e.g., Spark, Databricks), and data visualization tools (e.g., Tableau, Power BI). Proven ability to lead a team of engineers, fostering a collaborative and high-performing environment.
    $76k-100k yearly est. 5d ago
  • Sr. Data Engineer (SQL+Python+AWS)

    SGS Technologie 3.5company rating

    Saint Petersburg, FL jobs

    looking for a Sr. Data Engineer (SQL+Python+AWS) to work on a 12+ Months, Contract (potential Extension or may Convert to Full-time) = Hybrid at St. Petersburg, FL 33716 with a Direct Financial Client = only on W2 for US Citizen or Green Card Holders. Notes from the Hiring Manager: • Setting up Python environments and data structures to support the Data Science/ML team. • No prior Data Science or Machine Learning experience required. • Role involves building new data pipelines and managing file-loading connections. • Strong SQL skills are essential. • Contract-to-hire position. • Hybrid role based in St. Pete, FL (33716) only. Duties: This role is building and maintaining data pipelines that connect Oracle-based source systems to AWS cloud environments, to provide well-structured data for analysis and machine learning in AWS SageMaker. It includes working closely with data scientists to deliver scalable data workflows as a foundation for predictive modeling and analytics. • Develop and maintain data pipelines to extract, transform, and load data from Oracle databases and other systems into AWS environments (S3, Redshift, Glue, etc.). • Collaborate with data scientists to ensure data is prepared, cleaned, and optimized for SageMaker-based machine learning workloads. • Implement and manage data ingestion frameworks, including batch and streaming pipelines. • Automate and schedule data workflows using AWS Glue, Step Functions, or Airflow. • Develop and maintain data models, schemas, and cataloging processes for discoverability and consistency. • Optimize data processes for performance and cost efficiency. • Implement data quality checks, validation, and governance standards. • Work with DevOps and security teams to comply with RJ standards. Skills: Required: • Strong proficiency with SQL and hands-on experience working with Oracle databases. • Experience designing and implementing ETL/ELT pipelines and data workflows. • Hands-on experience with AWS data services, such as S3, Glue, Redshift, Lambda, and IAM. • Proficiency in Python for data engineering (pandas, boto3, pyodbc, etc.). • Solid understanding of data modeling, relational databases, and schema design. • Familiarity with version control, CI/CD, and automation practices. • Ability to collaborate with data scientists to align data structures with model and analytics requirements Preferred: • Experience integrating data for use in AWS SageMaker or other ML platforms. • Exposure to MLOps or ML pipeline orchestration. • Familiarity with data cataloging and governance tools (AWS Glue Catalog, Lake Formation). • Knowledge of data warehouse design patterns and best practices. • Experience with data orchestration tools (e.g., Apache Airflow, Step Functions). • Working knowledge of Java is a plus. Education: B.S. in Computer Science, MIS or related degree and a minimum of five (5) years of related experience or combination of education, training and experience.
    $71k-91k yearly est. 3d ago
  • Senior Data Engineer

    Zillion Technologies, Inc. 3.9company rating

    McLean, VA jobs

    The candidate must have 5+ years of hands on experience working with PySpark/Python, microservices architecture, AWS EKS, SQL, Postgres, DB2, Snowflake, Behave OR Cucumber frameworks, Pytest (unit testing), automation testing and regression testing. Experience with tools such as Jenkins, SonarQube AND/OR Fortify are preferred for this role. Experience in Angular and DevOps are nice to haves for this role. Must Have Qualifications: PySpark/Python based microservices, AWS EKS, Postgres SQL Database, Behave/Cucumber for automation, Pytest, Snowflake, Jenkins, SonarQube and Fortify. Responsibilities: Development of microservices based on Python, PySpark, AWS EKS, AWS Postgres for a data-oriented modernization project. New System: Python and PySpark, AWS Postgres DB, Behave/Cucumber for automation, and Pytest Perform System, functional and data analysis on the current system and create technical/functional requirement documents. Current System: Informatica, SAS, AutoSys, DB2 Write automated tests using Behave/cucumber, based on the new micro-services-based architecture Promote top code quality and solve issues related to performance tuning and scalability. Strong skills in DevOps, Docker/container-based deployments to AWS EKS using Jenkins and experience with SonarQube and Fortify. Able to communicate and engage with business teams and analyze the current business requirements (BRS documents) and create necessary data mappings. Preferred strong skills and experience in reporting applications development and data analysis Knowledge in Agile methodologies and technical documentation.
    $77k-109k yearly est. 5d ago
  • Senior Snowflake Data Engineer

    Zensar Technologies 4.3company rating

    Santa Clara, CA jobs

    About the job Why Zensar? We're a bunch of hardworking, fun-loving, people-oriented technology enthusiasts. We love what we do, and we're passionate about helping our clients thrive in an increasingly complex digital world. Zensar is an organization focused on building relationships, with our clients and with each other-and happiness is at the core of everything we do. In fact, we're so into happiness that we've created a Global Happiness Council, and we send out a Happiness Survey to our employees each year. We've learned that employee happiness requires more than a competitive paycheck, and our employee value proposition-grow, own, achieve, learn (GOAL)-lays out the core opportunities we seek to foster for every employee. Teamwork and collaboration are critical to Zensar's mission and success, and our teams work on a diverse and challenging mix of technologies across a broad industry spectrum. These industries include banking and financial services, high-tech and manufacturing, healthcare, insurance, retail, and consumer services. Our employees enjoy flexible work arrangements and a competitive benefits package, including medical, dental, vision, 401(k), among other benefits. If you are looking for a place to have an immediate impact, to grow and contribute, where we work hard, play hard, and support each other, consider joining team Zensar! Zensar is seeking an Senior Snowflake Data Engineer -Santa Clara, CA-Work from office all 5 days-This is open for Full time with excellent benefits and growth opportunities and contract role as well. Job Description: Key Requirements: Strong hands-on experience in data engineering using Snowflake and Databricks, with proven ability to build and optimize large-scale data pipelines. Deep understanding of data architecture principles, including ingestion, transformation, storage, and access control. Solid experience in system design and solution architecture, focusing on scalability, reliability, and maintainability. Expertise in ETL/ELT pipeline design, including data extraction, transformation, validation, and load processes. In-depth knowledge of data modeling techniques (dimensional modeling, star, and snowflake schemas). Skilled in optimizing compute and storage costs across Snowflake and Databricks environments. Strong proficiency in administration, including database design, schema management, user roles, permissions, and access control policies. Hands-on experience implementing data lineage, quality, and monitoring frameworks. Advanced proficiency in SQL for data processing, transformation, and automation. Experience with reporting and visualization tools such as Power BI and Sigma Computing. Excellent communication and collaboration skills, with the ability to work independently and drive technical initiatives. Zensar believes that diversity of backgrounds, thought, experience, and expertise fosters the robust exchange of ideas that enables the highest quality collaboration and work product. Zensar is an equal opportunity employer. All employment decisions shall be made without regard to age, race, creed, color, religion, sex, national origin, ancestry, disability status, veteran status, sexual orientation, gender identity or expression, genetic information, marital status, citizenship status or any other basis as protected by federal, state, or local law. Zensar is committed to providing veteran employment opportunities to our service men and women. Zensar is committed to providing equal employment opportunities for persons with disabilities or religious observances, including reasonable accommodation when needed. Accommodations made to facilitate the recruiting process are not a guarantee of future or continued accommodations once hired. Zensar does not facilitate/sponsor any work authorization for this position. Candidates who are currently employed by a client or vendor of Zensar may be ineligible for consideration. Zensar values your privacy. We'll use your data in accordance with our privacy statement located at: *********************************
    $109k-150k yearly est. 3d ago
  • Python Data Engineer- THADC5693417

    Compunnel Inc. 4.4company rating

    Houston, TX jobs

    Must Haves: Strong proficiency in Python; 5+ years' experience. Expertise in Fast API and microservices architecture and coding Linking python based apps with sql and nosql db's Deployments on docker, Kubernetes and monitoring tools Experience with Automated testing and test-driven development Git source control, git actions, ci/cd , VS code and copilot Expertise in both on prem sql dbs (oracle, sql server, Postgres, db2) and no sql databases Working knowledge of data warehousing and ETL Able to explain the business functionality of the projects/applications they have worked on Ability to multi task and simultaneously work on multiple projects. NO CLOUD - they are on prem Day to Day: Insight Global is looking for a Python Data Engineer for one of our largest oil and gas clients in Downtown Houston, TX. This person will be responsible for building python-based relationships between back-end SQL and NoSQL databases, architecting and coding Fast API and Microservices, and performing testing on back-office applications. The ideal candidate will have experience developing applications utilizing python and microservices and implementing complex business functionality utilizing python.
    $78k-101k yearly est. 4d ago

Learn more about ZP Group jobs

View all jobs