These network interfaces then provide network connectivity for AWS Glue through your VPC. AWS Glue creates elastic network interfaces (ENIs) in a VPC/private subnet.
The JDBC connection defines parameters for a data store-for example, the JDBC connection to the PostgreSQL server running on an on-premises network. Network connectivity exists between the Amazon VPC and the on-premises network using a virtual private network (VPN) or AWS Direct Connect (DX). The solution architecture illustrated in the diagram works as follows: The ENIs in the VPC help connect to the on-premises database server over a virtual private network (VPN) or AWS Direct Connect (DX). The solution uses JDBC connectivity using the elastic network interfaces (ENIs) in the Amazon VPC. The following diagram shows the architecture of using AWS Glue in a hybrid environment, as described in this post. AWS Glue jobs extract data, transform it, and load the resulting data back to S3, data stores in a VPC, or on-premises JDBC data stores as a target. AWS Glue can also connect to a variety of on-premises JDBC data stores such as PostgreSQL, MySQL, Oracle, Microsoft SQL Server, and MariaDB.ĪWS Glue ETL jobs can use Amazon S3, data stores in a VPC, or on-premises JDBC data stores as a source. For more information, see Adding a Connection to Your Data Store. AWS Glue can connect to Amazon S3 and data stores in a virtual private cloud (VPC) such as Amazon RDS, Amazon Redshift, or a database running on Amazon EC2. 
In this post, I describe a solution for transforming and moving data from an on-premises data store to Amazon S3 using AWS Glue that simulates a common data lake ingestion pipeline. For optimal operation in a hybrid environment, AWS Glue might require additional network, firewall, or DNS configuration. AWS Glue ETL jobs can interact with a variety of data sources inside and outside of the AWS environment. May 2022: This post was reviewed for accuracy.ĪWS Glue is a fully managed ETL (extract, transform, and load) service to catalog your data, clean it, enrich it, and move it reliably between various data stores.
0 kommentar(er)
