#SGUnitedJobs Data Engineer (D Cube)
GOVERNMENT TECHNOLOGY AGENCY
Information Technology, Public / Civil Service
The Government Digital Services (GDS) Team aims to spearhead the digital transformation of government. Our team aims to design and develop software applications that help government agencies to better serve the needs of the people of Singapore.
We are looking for data engineers that are open and enthusiastic about end to end product delivery and trying out new technologies. Be part of a cross-functional development team, that values quality, automation and user experience of the product we deliver.
What to Expect:
- Design, build, launch and maintain efficient and reliable large-scale batch and real-time data pipelines with data processing frameworks
- Integrate and collate data silos in a manner which is both scalable and compliant
- Collaborate with Project Manager, Frontend Developers, UX Designers and Data Analyst to build scalable data-driven products
- Responsible for developing backend APIs & working on databases to support the applications
- Working in an Agile Environment that practices Continuous Integration and Delivery
- Working closely with fellow developers through pair programming and code review process
- Driving enterprise data foundation requirements of Data Warehousing, Data Lake
- Acquiring, storing, governing and processing large datasets of structured/unstructured data
- Identifying and implementing the right architecture components for specific tasks
- Monitoring and Optimising infrastructure and pipeline
- Implementing and improving ETL / ELT / Streaming big data pipelines
- Ensuring Data Quality through continuous improvement and monitoring
- Implementing Architecture-as-code and Data-pipeline-as-code best practises
How to Succeed:
- You hold a Bachelor or Master in IT, Information Management and/or Computer Science
- Good knowledge of big data technology landscape and concepts related to distributed storage and computing
- Experience with big data processing tools such as Spark, Flink, Samza, Beam, etc.
- Experience with batch and ETL jobs to ingest and process data
- Experience with Data Warehouses such as Redshift, BigQuery, Snowflake, etc.
- Experience with Cloud environments such as AWS, GCP, Azure
- Experience with other NoSQL databases such as Elasticsearch, DynamoDB, Cassandra, etc.
- Programming experience with SQL, Python, Java, Scala
- Experience with event sourcing systems such as Kafka, Kinesis and the associated APIs such as Kafka Connect, Kafka Streams, KCL, Spark Structured Streaming, etc.
- Experience or willingness to work on DevOps practises such as infrastructure-as-code, data-pipeline-as-code
- Understanding of Events and Time-series Data-modelling
- High-level understanding of Data-science model development topics such as training and deployment
- You are passionate about technology and are always looking to improve yourself
- Interested in being the bridge between engineering and analytics
- Knowledgeable about system design, data structure and algorithms
Closing on 14 Jul 2021