Data Engineer

Company: SoundTransit
Location: Seattle
Posted on: June 24, 2022

Job Description:

DescriptionImportant information pertaining to our COVID-19 protocols, including our hybrid workforce options, can be found .
GENERAL PURPOSE:
Under minimal supervision, designs, builds, tests, and delivers, end-to-end, automated data pipelines over complex on-prem and off-prem platforms. Automates the collection of data from multiple source systems containing structured, semi-structured and unstructured data; integrates, transforms, organizes, stores, and serves data to data analysts, data scientists, and other data consumers within the agency departments. Possesses excellent programming, debugging, testing and problem-solving skills. Leverages pragmatic data management practices and principles to ensure coherent movement of data that are relied upon across the agency as the single source of truth.
ESSENTIAL FUNCTIONS:
The following duties are a representative summary of the primary duties and responsibilities. Incumbent(s) may not be required to perform all duties listed and may be required to perform additional, position-specific duties.
* Analyzes business requirements, designs, and implements enterprise-class data solutions for new data projects.
* Develops and operationalizes data pipelines to make data available for consumption
* Owns the design, development, and ongoing maintenance of data warehouses, data marts, multi-dimensional cubes, and data lakes.
* Writes complex SQL queries, stored procedures, ETL workflows, views, temporary tables, and other database objects.
* Utilizes latest data technologies and cloud platforms; collects, integrates, ingests, transforms, and organizes, and serves data and information to support data analyses, reporting, dashboarding and modern analytics across the agency.
* Works in tandem with tech leads, data architects and peer data engineers to design data pipelines and recommends ongoing optimization of, data ingestion, data quality, data storage and orchestration.
* Evaluates existing legacy data solutions; redesigns and implements modern data infrastructure for robustness and greater scalability.
* Assists customers with their data related technical issues and data platform needs.
* Builds analytics tools that ingest data from large disparate datasets to draw actionable insights into operational efficiencies and key business performance metrics.
* Identifies ways to improve data reliability and efficiency.
* Develops automated test scripts to monitor and report on data quality issues; conducts root cause analyses in response to data issues and implements cost effective resolutions for data anomalies.
* Collaborates with the IT Enterprise Architecture team to define data architecture framework and implement data solutions and toolsets.
* Provides input into data governance initiatives and influences the development of sustainable data management and governance practices across the agency.
* Creates and maintains architecture diagrams, system documentation, data models, mapping documents, business rules, data flow diagrams and other design related artifacts.
* Actively participates in architecture design, and code review sessions with peers, solution architects, consultants, and vendors.
* Attends professional group meetings; stay abreast of new trends and innovations in the field of information technology.
* Conducts research and makes recommendations on emerging technologies, toolsets, applications, and systems, considering among other factors cost savings, adoption, standardization/simplification, flexibility, and reuse.
* Troubleshoots and resolves production issues quickly to ensure system uptime meets service level agreements.
* Performs peer reviews for other data engineer's work.
* Follows agency's data classification policies and procedures, and maintain compliance with established security standards and best practices
* Communicates statuses of all assigned tasks clearly and concisely to the project team, stakeholders, and management.
* Works cooperatively with coworkers, product managers, customers, project managers and vendors.
* Champions and models Sound Transit's core values and demonstrates values-based behaviors in everyday interactions across the agency.
* Contributes to a culture of diversity, equity and inclusion in alignment with Sound Transit's Equity & Inclusion Policy.
* It is the responsibility of all employees to follow the Agency safety rules, regulations, and procedures pertaining to their assigned duties and responsibilities, which could include systems, operations, and/or other employees.
* It is the responsibility of all employees to integrate sustainability into everyday business practices.
* Other duties as assigned.
MINIMUM QUALIFICATIONS:
Education and Experience:
Bachelor's Degree in computer science, information technology, engineering, or closely related field and five years of experience in information technology in the areas of data engineering, business intelligence and analytics; OR an equivalent combination of education and experience.
Required Licenses or Certifications:
None
Required Knowledge and Skills:
* Basic principles of computer science, database technologies and information systems.
* Advanced principles, best practices, methods, and techniques used in data engineering, business intelligence, data management, data warehousing concepts, data mapping, data modeling, reporting, analytics, and data science.
* Relational database management systems, NoSQL databases and Big Data ecosystem
* Data pipelines, stream processing, Supervisory Control And Data Acquisition (SCADA) and Internet of Things (IoT).
* Enterprise data catalog, meta-data and master-data concepts and management.
* Full-stack DevOps engineering with understanding of compute, network, storage with cost optimized implementations.
* DevOps tools like ADO, Git, Jenkins, Dockers etc.
* Enterprise application systems related to Finance, Human Resources, Asset Management, Transit Rider Technologies, and other subject areas as needed.
* Evaluating business requirements and developing information technology solutions.
* Operational characteristics of a variety of computer and network systems, applications, hardware, software, and peripheral equipment, including enterprise business systems.
* Various methods of service delivery including agile and waterfall.
* Software development life cycle, source code version control systems, release management, change management, and ITIL processes (Incident and Change Management).
* Principles, practices, methods, and techniques used in the installation, troubleshooting, and maintenance of software systems and applications.
* Quality Assurance techniques and automated testing practices.
* Principles and practices of project management.
* Pertinent federal, state, and local laws, codes, and regulations.
* Designing and implementing Business Intelligence/Analytics platforms using Azure services such as Azure Synapse Analytics, Azure Data Factory, Azure Data Lake to improve and speed up delivery of data services.
* Relational Database Management Systems (RDBMS), Microsoft SQL Server, SQL Server Integration Services (SSIS), SQL Server Analysis Services (SSAS), and other relevant technologies.
* Message based integration using Kafka and/or Azure Event Hub
* Logical and physical data modeling for both OLTP and OLAP.
* Reporting technologies including but are not limited to Microsoft SQL Server Reporting Services (SSRS), Microsoft Report Builder, Power BI, Crystal Reports, Business Objects Enterprise, etc.
* Working knowledge and advanced skills in Excel as a data source and reporting tool.
* Big Data tools like Spark, MapReduce, Hive, Pig, Oozie, etc.
* NoSQL solutions like HBase, MongoDB, Cassandra etc.
* Programming using C#, Java, Scala, and Python; scripting using PowerShell.
* Creating and consuming various data formats such as CSV, XML, JSON, etc. including using of APIs or web services for data acquisition.
* Learning new applications, programming languages, development tools, and technologies quickly.
* Establishing and maintaining effective working relationships with peers, other department staff, management, vendors, outside agencies, community groups, external business partners and the public.
* Effective oral and written communication and the ability to convey technical details to non-technical stakeholders, senior leadership, and executives both in written and verbal form
* Working effectively under pressure, meeting tight deadlines, and adjusting to changing priorities.
Physical Demands / Work Environment:
* Work is performed in a standard office environment.
* Subject to standing, walking, bending, reaching, stooping, and lifting of objects up to 40 pounds; may occasionally be exposed to dangerous machinery, extreme temperatures, and extreme noise when working in the data center environment or when working in the field.
* The Agency promotes a safe and healthy work environment and provides appropriate safety and equipment training for all personnel as required.
Sound Transit is an equal employment opportunity employer. No person is unlawfully excluded from employment action based on race, color, religion, national origin, sex (including gender identity, sexual orientation and pregnancy), age, genetic information, disability, veteran status or other protected class.

Equal Opportunity Employer/Protected Veterans/Individuals with DisabilitiesThe contractor will not discharge or in any other manner discriminate against employees or applicants because they have inquired about, discussed, or disclosed their own pay or the pay of another employee or applicant. However, employees who have access to the compensation information of other employees or applicants as a part of their essential job functions cannot disclose the pay of other employees or applicants to individuals who do not otherwise have access to compensation information, unless the disclosure is (a) in response to a formal complaint or charge, (b) in furtherance of an investigation, proceeding, hearing, or action, including an investigation conducted by the employer, or (c) consistent with the contractor's legal duty to furnish information. 41 CFR 60-1.35(c)

Keywords: SoundTransit, Seattle , Data Engineer, Engineering , Seattle, Washington