Data Engineer
Company: SoundTransit
Location: Seattle
Posted on: June 24, 2022
|
|
Job Description:
DescriptionImportant information pertaining to our COVID-19
protocols, including our hybrid workforce options, can be found
.
GENERAL PURPOSE:
Under minimal supervision, designs, builds, tests, and delivers,
end-to-end, automated data pipelines over complex on-prem and
off-prem platforms. Automates the collection of data from multiple
source systems containing structured, semi-structured and
unstructured data; integrates, transforms, organizes, stores, and
serves data to data analysts, data scientists, and other data
consumers within the agency departments. Possesses excellent
programming, debugging, testing and problem-solving skills.
Leverages pragmatic data management practices and principles to
ensure coherent movement of data that are relied upon across the
agency as the single source of truth.
ESSENTIAL FUNCTIONS:
The following duties are a representative summary of the primary
duties and responsibilities. Incumbent(s) may not be required to
perform all duties listed and may be required to perform
additional, position-specific duties.
* Analyzes business requirements, designs, and implements
enterprise-class data solutions for new data projects.
* Develops and operationalizes data pipelines to make data
available for consumption
* Owns the design, development, and ongoing maintenance of data
warehouses, data marts, multi-dimensional cubes, and data
lakes.
* Writes complex SQL queries, stored procedures, ETL workflows,
views, temporary tables, and other database objects.
* Utilizes latest data technologies and cloud platforms; collects,
integrates, ingests, transforms, and organizes, and serves data and
information to support data analyses, reporting, dashboarding and
modern analytics across the agency.
* Works in tandem with tech leads, data architects and peer data
engineers to design data pipelines and recommends ongoing
optimization of, data ingestion, data quality, data storage and
orchestration.
* Evaluates existing legacy data solutions; redesigns and
implements modern data infrastructure for robustness and greater
scalability.
* Assists customers with their data related technical issues and
data platform needs.
* Builds analytics tools that ingest data from large disparate
datasets to draw actionable insights into operational efficiencies
and key business performance metrics.
* Identifies ways to improve data reliability and efficiency.
* Develops automated test scripts to monitor and report on data
quality issues; conducts root cause analyses in response to data
issues and implements cost effective resolutions for data
anomalies.
* Collaborates with the IT Enterprise Architecture team to define
data architecture framework and implement data solutions and
toolsets.
* Provides input into data governance initiatives and influences
the development of sustainable data management and governance
practices across the agency.
* Creates and maintains architecture diagrams, system
documentation, data models, mapping documents, business rules, data
flow diagrams and other design related artifacts.
* Actively participates in architecture design, and code review
sessions with peers, solution architects, consultants, and
vendors.
* Attends professional group meetings; stay abreast of new trends
and innovations in the field of information technology.
* Conducts research and makes recommendations on emerging
technologies, toolsets, applications, and systems, considering
among other factors cost savings, adoption,
standardization/simplification, flexibility, and reuse.
* Troubleshoots and resolves production issues quickly to ensure
system uptime meets service level agreements.
* Performs peer reviews for other data engineer's work.
* Follows agency's data classification policies and procedures, and
maintain compliance with established security standards and best
practices
* Communicates statuses of all assigned tasks clearly and concisely
to the project team, stakeholders, and management.
* Works cooperatively with coworkers, product managers, customers,
project managers and vendors.
* Champions and models Sound Transit's core values and demonstrates
values-based behaviors in everyday interactions across the
agency.
* Contributes to a culture of diversity, equity and inclusion in
alignment with Sound Transit's Equity & Inclusion Policy.
* It is the responsibility of all employees to follow the Agency
safety rules, regulations, and procedures pertaining to their
assigned duties and responsibilities, which could include systems,
operations, and/or other employees.
* It is the responsibility of all employees to integrate
sustainability into everyday business practices.
* Other duties as assigned.
MINIMUM QUALIFICATIONS:
Education and Experience:
Bachelor's Degree in computer science, information technology,
engineering, or closely related field and five years of experience
in information technology in the areas of data engineering,
business intelligence and analytics; OR an equivalent combination
of education and experience.
Required Licenses or Certifications:
None
Required Knowledge and Skills:
* Basic principles of computer science, database technologies and
information systems.
* Advanced principles, best practices, methods, and techniques used
in data engineering, business intelligence, data management, data
warehousing concepts, data mapping, data modeling, reporting,
analytics, and data science.
* Relational database management systems, NoSQL databases and Big
Data ecosystem
* Data pipelines, stream processing, Supervisory Control And Data
Acquisition (SCADA) and Internet of Things (IoT).
* Enterprise data catalog, meta-data and master-data concepts and
management.
* Full-stack DevOps engineering with understanding of compute,
network, storage with cost optimized implementations.
* DevOps tools like ADO, Git, Jenkins, Dockers etc.
* Enterprise application systems related to Finance, Human
Resources, Asset Management, Transit Rider Technologies, and other
subject areas as needed.
* Evaluating business requirements and developing information
technology solutions.
* Operational characteristics of a variety of computer and network
systems, applications, hardware, software, and peripheral
equipment, including enterprise business systems.
* Various methods of service delivery including agile and
waterfall.
* Software development life cycle, source code version control
systems, release management, change management, and ITIL processes
(Incident and Change Management).
* Principles, practices, methods, and techniques used in the
installation, troubleshooting, and maintenance of software systems
and applications.
* Quality Assurance techniques and automated testing practices.
* Principles and practices of project management.
* Pertinent federal, state, and local laws, codes, and
regulations.
* Designing and implementing Business Intelligence/Analytics
platforms using Azure services such as Azure Synapse Analytics,
Azure Data Factory, Azure Data Lake to improve and speed up
delivery of data services.
* Relational Database Management Systems (RDBMS), Microsoft SQL
Server, SQL Server Integration Services (SSIS), SQL Server Analysis
Services (SSAS), and other relevant technologies.
* Message based integration using Kafka and/or Azure Event Hub
* Logical and physical data modeling for both OLTP and OLAP.
* Reporting technologies including but are not limited to Microsoft
SQL Server Reporting Services (SSRS), Microsoft Report Builder,
Power BI, Crystal Reports, Business Objects Enterprise, etc.
* Working knowledge and advanced skills in Excel as a data source
and reporting tool.
* Big Data tools like Spark, MapReduce, Hive, Pig, Oozie, etc.
* NoSQL solutions like HBase, MongoDB, Cassandra etc.
* Programming using C#, Java, Scala, and Python; scripting using
PowerShell.
* Creating and consuming various data formats such as CSV, XML,
JSON, etc. including using of APIs or web services for data
acquisition.
* Learning new applications, programming languages, development
tools, and technologies quickly.
* Establishing and maintaining effective working relationships with
peers, other department staff, management, vendors, outside
agencies, community groups, external business partners and the
public.
* Effective oral and written communication and the ability to
convey technical details to non-technical stakeholders, senior
leadership, and executives both in written and verbal form
* Working effectively under pressure, meeting tight deadlines, and
adjusting to changing priorities.
Physical Demands / Work Environment:
* Work is performed in a standard office environment.
* Subject to standing, walking, bending, reaching, stooping, and
lifting of objects up to 40 pounds; may occasionally be exposed to
dangerous machinery, extreme temperatures, and extreme noise when
working in the data center environment or when working in the
field.
* The Agency promotes a safe and healthy work environment and
provides appropriate safety and equipment training for all
personnel as required.
Sound Transit is an equal employment opportunity employer. No
person is unlawfully excluded from employment action based on race,
color, religion, national origin, sex (including gender identity,
sexual orientation and pregnancy), age, genetic information,
disability, veteran status or other protected class.
Equal Opportunity Employer/Protected Veterans/Individuals with
DisabilitiesThe contractor will not discharge or in any other
manner discriminate against employees or applicants because they
have inquired about, discussed, or disclosed their own pay or the
pay of another employee or applicant. However, employees who have
access to the compensation information of other employees or
applicants as a part of their essential job functions cannot
disclose the pay of other employees or applicants to individuals
who do not otherwise have access to compensation information,
unless the disclosure is (a) in response to a formal complaint or
charge, (b) in furtherance of an investigation, proceeding,
hearing, or action, including an investigation conducted by the
employer, or (c) consistent with the contractor's legal duty to
furnish information. 41 CFR 60-1.35(c)
Keywords: SoundTransit, Seattle , Data Engineer, Engineering , Seattle, Washington
Click
here to apply!
|