Perform CRUD operations on a MySQL database

This tutorial shows you how to connect to a MySQL database instance from a sample integration and perform List, Get, Create, Update, and Delete operations on a MySQL database table.

To complete this tutorial, perform the following tasks:

• Set up a database and table in your MySQL server
• Create a MySQL connection
• Configure an Integration to use the MySQL connection
• Test the Integration
• Perform other operations on your MySQL database table

Before you begin

• Ensure that you have access to integrations.

• Do the following in your Google Cloud project:

  • Grant the following roles to the service account that you want to use to create the connection:
    • roles/secretmanager.viewer
    • roles/secretmanager.secretAccessor
  • Enable the following services:
    • secretmanager.googleapis.com (Secret Manager API)
    • connectors.googleapis.com (Connectors API)

    If these services have not been enabled for your project previously, you are prompted to enable them when creating the connection in the Create Connection page.

• Ensure that you have access to a MySQL server that you can use to create a database.

Set up a database and table in your MySQL server

Connect to your MySQL server and create a database and a table to use in this tutorial.

1. To connect to your MySQL server, execute the following command from a system that has a MySQL client installed:

   mysql --host=MySQL server host name or IP address --port=MySQL server port number -uusername -ppassword

   In this example, replace:
   • MySQL server host name or IP address with the name or IP address of your MySQL server.
   • MySQL server port number with the port number for your MySQL server.
   • username with the username for your MySQL server.
   • password with the password for your MySQL server.
2. To create a MySQL database to use in this tutorial, execute the following command from your MySQL client:

   CREATE DATABASE tutorialDB;

3. To create a table to use in this tutorial, execute the following command from your MySQL client:

   create table employee  (  employee_id int auto_increment primary key,  employee_first_name varchar(500) NOT null,  employee_last_name varchar(500) NOT null,  employee_emailID varchar(500)  );

4. To add rows to theemployee table that you created, execute the following command from your MySQL client:

   INSERT INTO employee (employee_first_name,employee_last_name,employee_emailID) values ("Peter","Dilliard","test-01@test.com");INSERT INTO employee (employee_first_name,employee_last_name,employee_emailID) values ("Meaghan","Webb","test-02@test.com");

5. Verify that the table is created and rows are added by executing the following command:

   SELECT * FROM employee;

   The following table rows are displayed:

   +-------------+---------------------+--------------------+------------------+| employee_id | employee_first_name | employee_last_name | employee_emailID |+-------------+---------------------+--------------------+------------------+|           1 | Peter               | Dilliard           | test-01@test.com ||           2 | Meaghan             | Webb               | test-02@test.com |+-------------+---------------------+--------------------+------------------+

Create a MySQL connection

To enable an integration to connect to your MySQL database, create a new connection to your MySQL database:

1. Click+Create new to open theCreate Connection page.
2. In theLocation section, choose the location for the connection.
   1. From theRegion list, select the region where you want to create the connection.

      For the list of all the supported regions, seeLocations.

   2. ClickNext.
3. In theConnection Details section, enter the following details:
   1. From theConnector list, selectMySQL.
   2. From theConnector version list, select the connector version.
   3. In theConnection Name field, enter a name for the connection instance.

      Connection names must meet the following criteria:

      • Use letters, numbers, or hyphens.
      • Letters must be lower-case.
      • Names must begin with a letter and end with a letter or number.
      • Names cannot exceed 49 characters.
   4. Optional: In theDescription field, add a description of the connection instance.
   5. Optional: EnableCloud Logging.
   6. From theService Account list, select a service account that has therequired roles.
   7. In theDatabase name field, enter the name of the MySQL database.
   8. Optionally, configure theConnection node settings:
      • Minimum number of nodes: Enter the minimum number of connection nodes.
      • Maximum number of nodes: Enter the maximum number of connection nodes.

      A node is a unit (or replica) of a connection that processes transactions. More nodes are required to process more transactions for a connection and conversely, fewer nodes are required to process fewer transactions. To understand how the nodes affect your connector pricing, see Pricing for connection nodes. If you don't enter any values, by default the minimum nodes are set to 2 (for better availability) and the maximum nodes are set to 50.

   Note: You can customize the connection node values only if you are a Pay-as-you-go customer.
   9. (Optional) In theAdvanced settings section, select theUse proxy checkbox to configure a proxy server for the connection and configure the following values:
   • Proxy Auth Scheme: Select the authentication type to authenticate with the proxy server. The following authentication types are supported:
     • Basic: Basic HTTP authentication.
     • Digest: Digest HTTP authentication.
   • Proxy User: A user name to be used to authenticate with the proxy server.
   • Proxy Password: The Secret manager secret of the user's password.
   • Proxy SSL Type: The SSL type to use when connecting to the proxy server. The following authentication types are supported:
     • Auto: Default setting. If the URL is an HTTPS URL, then the Tunnel option is used. If the URL is an HTTP URL, then the NEVER option is used.
     • Always: The connection is always SSL enabled.
     • Never: The connection is not SSL enabled.
     • Tunnel: The connection is through a tunneling proxy. The proxy server opens a connection to the remote host and traffic flows back and forth through the proxy.
   • In theProxy Server section, enter details of the proxy server.
     1. Click+ Add destination.
     2. Select aDestination Type.
        • Host address: Specify the hostname or IP address of the destination.

          If you want to establish a private connection to your backend system, do the following:

          • Create aPSC service attachment.
          • Create anendpoint attachment and then enter the details of the endpoint attachment in theHost address field.
   10. Optional: To add a label to the connection, click+Add Label.
   11. ClickNext.
   12. In theDestinations section, enter details of the remote host (backend system) to which you want to connect.
       • From theDestination Type list, select a host address.
         • To specify the destination hostname or IP address, selectHost address and enter the address in theHost 1 field.
         • To establish a private connection, selectHost Address and add the endpoint attachment created for the SAP Gateway using the HTTPS protocol.
         Note:To understand how to create an endpoint attachment, seePSC service attachment andendpoint attachment. After you have created the endpoint attachment, it will be visible in theEndpoint Attachment list.

         If you want to establish a public connection to your backend systems with additional security, you can considerconfiguring static outbound IP addresses for your connections, and then configure your firewall rules to allowlist only the specific static IP addresses.

         To enter additional destinations, click+Add Destination.

       • ClickNext.
   13. In theAuthentication section, you can provide credentials:
       • In theUsername field, enter the MySQL username for the connection.
       • In thePassword field, enter theSecret Manager secret containing the password associated with the MySQL username.
         • If you have previously created a secret, and it is not available in the list, selectEnter Secret Manually. In theAdd a secret by resource ID dialog, copy and paste the resource ID from theSecret Manager.
           • To use the latest version, copy and paste the resource ID from the parent secret, in the format:"projects/project-number/secrets/secret-name"
           • To select a specific version, copy and paste the resource ID for that specific version, in the format"projects/project-number/secrets/secret-name/versions/1"

           To add the secret, clickAdd Secret.

         • If you have not created a secret for use with MySQL, clickCreate New Secret. In theCreate Secret dialog enter the following details:
           • In theName field, enter the secret name.
           • In theSecret value field, enter the contents of the secret or upload a file that contains the secret.
           • ClickCreate Secret.
       • From theSecret version list, select the version of thePassword secret from the list of available versions in the drop-down.
       • ClickNext.
4. In theReview section, review your connection and authentication details.
5. ClickCreate.

Configure an Integration to use the MySQL connection

To use the MySQL connection that you created in an integration, add aConnectors task in an integration along with anAPI Trigger. TheAPI Trigger is connected to theConnectors task using anEdge connection.

Create a new Integration

1. In the Google Cloud console, go to theIntegration Connectors page.

   Go to Integration Connectors

2. In the navigation menu, clickIntegrations.

   TheIntegrations List page appears.

3. Select an existing integration or create a new integration by clickingCREATE INTEGRATION.

   This opens the integration in the integration editor page.

4. In the integration editor, click+Add a task/trigger > Tasks to view the list of available tasks.
5. ClickCREATE NEW.
6. Enter a name and (optionally) a description in theCreate Integration dialog.Note: Integration names must meet the following requirements:
   • Names must start and end with letters or numbers.
   • Names cannot contain spaces or more than one dash or underscore in a row.
7. ClickCreate to open the integration editor.

Add and configure an API trigger

To add and configure anAPI trigger to the integration, do the following:

1. In the integration editor, selectAdd a task/trigger > Triggers to display a list of available triggers.
2. Drag theAPI Trigger element to the integration editor.

Add and configure a Connectors task

Perform the following steps to configure aConnectors task to list all the entities in theemployee table:

1. Select+Add a task/trigger > Tasks in the integration editor to display the list of available tasks.
2. Drag theConnectors element to the integration editor.
3. Click theConnectors task element on the designer to view the task configuration pane.
4. ClickConfigure task.

   TheConfigure connector task dialog appears.

5. In theConfigure connector task dialog, do the following:
   1. Select the connection region where you created your MySQL connection.
   2. Once a region is chosen, theConnection column appears. Select the MySQL connection that you created from the list of available connections.
   3. Once a connection is chosen, theType column appears. SelectEntities and thenemployee from the list of available entities.
   4. Once a type is chosen, theOperation column appears. SelectList.
   5. ClickDone to complete the connection configuration and close the dialog.

Connect the API Trigger element to the Connectors task element

Next, add an edge connection to connect the API trigger to the Connectors task. An edge connection is a connection between any two elements in an integration. For more information on edges and edge conditions, seeEdges.

To add the edge connection, click theFork control point at the bottom of the API Trigger element. Drag and drop the edge connection at theJoin control point at the top of the Connectors task element.

Test the Integration

To test the integration, do the following:

1. Click theTest button in the integration editor toolbar.
2. Change theExecution Deadline (in minutes) value, if you like, and then clickTest Integration.
3. Once the integration completes successfully, theTest Integration pane displays the messageIntegration execution succeeded. To view the logs, clickView logs.
4. InResponse Parameters, underConnector output payload, the following output is displayed:

   [ {    "employee_id": 1.0,    "employee_first_name": "Peter",    "employee_last_name": "Dilliard",    "employee_emailID": "test-01@test.com"  },  {    "employee_id": 2.0,    "employee_first_name": "Meaghan",    "employee_last_name": "Webb",    "employee_emailID": "test-02@test.com"  } ]

Perform other operations on your MySQL database table

When you configure a Connectors task in your integration, you can select any one of the following operations:

• List
• Get
• Create
• Update
• Delete

You've already used the List operation to view all the rows in theemployee table. In the following sections of this tutorial, you'll use the Get, Create, Update, and Delete operations to add, modify or remove rows from theemployee table.

Get a row from a table

If you know the entity ID (or primary key) of the row that you want to fetch, provide that value as input to a Get operation in your integration. The details returned by the Get operation are similar to the details returned by the List operation. However, while the List operation fetches details of all the table rows that match the specified query, the Get operation fetches details of only the table row that matches the specified entity ID.

Remember that, while the List operation by default returns all the rows in the table, the Get operation requires an entity ID as a search parameter. So, to use the Get operation, you must know the entity ID of the row that you want to fetch, or provide a default value for the entity ID.

To get details of a specified row, perform the following steps to configure theConnectors task that you created earlier:

1. Click theConnectors task element on the designer to view the task configuration pane.
2. ClickConfigure task.

   TheConfigure connector task dialog appears.

3. In theConfigure connector task dialog, in theOperation column, selectGet and then clickDone.
4. In the task configuration pane, underTask Input clickEntity ID.
5. In theConfigure Variable dialog, selectUse as an input to integration and clickSave.
6. Click theTest button in the integration editor toolbar.
7. Change theExecution Deadline (in minutes) value, if you like.
8. Enter the entity ID of the entity for which you want to fetch details. Enter2.
9. ClickTest Integration.
10. Once the integration completes successfully, theTest Integration pane displays the messageIntegration execution succeeded. To view the logs, clickView logs.
11. InResponse Parameters, underConnector output payload, the following output is displayed:

    [ {    "employee_id": 2.0,    "employee_first_name": "Meaghan",    "employee_last_name": "Webb",    "employee_emailID": "test-02@test.com"  } ]

12. You can verify that the information displayed here matches the information in the corresponding row in your MySQL table. To view this information in your MySQL table, execute the following command from your MySQL client:

    SELECT * FROM employee WHERE employee_id=2;

    The following table row is displayed:

    +-------------+---------------------+--------------------+------------------+| employee_id | employee_first_name | employee_last_name | employee_emailID |+-------------+---------------------+--------------------+------------------+|           2 | Meaghan             | Webb               | test-02@test.com |+-------------+---------------------+--------------------+------------------+

Add a row to a table

The Create operation allows you to add a row in a table. When you use the Create operation, you must provide all the values for the entity in the connector input payload.

To add a row using the Create operation, perform the following steps to configure theConnectors task that you created earlier:

1. Click theConnectors task element on the designer to view the task configuration pane.
2. ClickConfigure task.

   TheConfigure connector task dialog appears.

3. In theConfigure connector task dialog, in theOperation column, selectCreate and then clickDone.
4. In the task configuration pane, underTask Input clickConnector input payload.
5. In theConfigure Variable dialog, selectUse as an input to integration and clickSave.
6. Click theTest button in the integration editor toolbar.
7. Change theExecution Deadline (in minutes) value, if you like.
8. Enter the details for the entity that you want to create. For example, to add a new employee in theemployee table, enter the following JSON:

   {  "employee_first_name": "Mary",  "employee_last_name": "Smith",  "employee_emailID": "test-03@test.com"}

   Note: Don't specify the primary key,employee_id, as it is a read-only value that is generated when the row is added to the table.
9. ClickTest Integration.
10. Once the integration completes successfully, theTest Integration pane displays the messageIntegration execution succeeded. To view the logs, clickView logs.
11. InResponse Parameters, underConnector input payload, the payload that you provided is displayed:

    {  "employee_first_name": "Mary",  "employee_last_name": "Smith",  "employee_emailID": "test-03@test.com"}

    UnderConnector output payload, the entity ID is displayed as output:

    {  "employee_id": 3.0}

12. You can verify that the row is added in your MySQL table. To view this information in your MySQL table, execute the following command from your MySQL client:

    SELECT * FROM employee;

    The following table rows are displayed:

    +-------------+---------------------+--------------------+------------------+| employee_id | employee_first_name | employee_last_name | employee_emailID |+-------------+---------------------+--------------------+------------------+|           1 | Peter               | Dilliard           | test-01@test.com ||           2 | Meaghan             | Webb               | test-02@test.com ||           3 | Mary                | Smith              | test-03@test.com |+-------------+---------------------+--------------------+------------------+

Update a row in a table

Use the Update operation to make changes to the values in a table row. For example, you can use this operation to update the email ID of an employee in theemployee table. To specify the entity that you want to update, you can provide the entity ID as you did for the Get operation. Alternatively, you can use theFilter clause parameter to pass values on which to filter the table rows. This is useful if you want to make the same change in multiple rows based on specific search criteria.

To update a table row, perform the following steps to configure theConnectors task that you created earlier:

1. Click theConnectors task element on the designer to view the task configuration pane.
2. ClickConfigure task.

   TheConfigure connector task dialog appears.

3. In theConfigure connector task dialog, in theOperation column, selectUpdate and then clickDone.
4. In the task configuration pane, underTask Input clickConnector input payload.
5. In theConfigure Variable dialog, selectUse as an input to integration and clickSave.
6. Next, in the task configuration pane, underTask Input clickFilter clause.
7. In theConfigure Variable dialog, selectUse as an input to integration and clickSave.
8. Click theTest button in the integration editor toolbar.
9. Change theExecution Deadline (in minutes) value, if you like.
10. Enter the criteria that you want to filter by. For example, to find an employee with the email IDtest-03@test.com, enter:

    employee_emailID="test-03@test.com"

    Note: If you have multiple table rows that match the filter criteria, the update that you specify in the next step is made on all the matching rows.
11. Enter the values that you want to update. For example, to update the email ID of all the employees in theemployee table whose current email ID matches the filter clausetest-03@test.com, enter the following JSON:

    {  "employee_emailID": "msmith@test.com"}

    Note: You don't need to specify values for all the columns in the table. Enter the values that you want to update. The other values aren't modified.
12. ClickTest Integration.
13. Once the integration completes successfully, theTest Integration pane displays the messageIntegration execution succeeded. To validate that the entity was updated, use the Get operation to get the details of the specified entity.
14. You can verify that the row is updated in your MySQL table. To view this information in your MySQL table, execute the following command from your MySQL client:

    SELECT * FROM employee;

    The following table rows are displayed:

    +-------------+---------------------+--------------------+------------------+| employee_id | employee_first_name | employee_last_name | employee_emailID |+-------------+---------------------+--------------------+------------------+|           1 | Peter               | Dilliard           | test-01@test.com ||           2 | Meaghan             | Webb               | test-02@test.com ||           3 | Mary                | Smith              | msmith@test.com  |+-------------+---------------------+--------------------+------------------+

Delete a row in a table

You can use the Delete operation to delete one or more table rows. Provide the entity ID or use the filter clause to specify the rows that you want to delete. Remember that, if you use the filter clause to specify criteria for rows that you want to delete, it's possible to delete multiple rows that match the given filter. If you want to delete only one specific row, use the entity ID.

To delete a table row using the entity ID, perform the following steps to configure theConnectors task that you created earlier:

1. Click theConnectors task element on the designer to view the task configuration pane.
2. ClickConfigure task.

   TheConfigure connector task dialog appears.

3. In theConfigure connector task dialog, in theOperation column, selectDelete and then clickDone.
4. In the task configuration pane, underTask Input clickEntity ID.
5. In theConfigure Variable dialog, selectUse as an input to integration and clickSave.
6. Click theTest button in the integration editor toolbar.
7. Change theExecution Deadline (in minutes) value, if you like.
8. Enter the entity ID of the table row that you want to delete. Enter3.
9. ClickTest Integration.

10. Once the integration completes successfully, theTest Integration pane displays the messageIntegration execution succeeded. To validate that the table row was deleted, use the Get operation to get the details of the row that you deleted. This should return an error for the specified entity ID.

    You can also use the List operation and optionally provide a filter clause for the table row (or rows) that you deleted. This returns the list of rows remaining, which could be an empty list.

    For example, if you deleted the table row with entity ID 3, then if you specify the filter clauseemployee_emailID="msmith@test.com" or the entity ID3 for the List operation, the response parameters in the logs display[].

11. You can verify that the row is deleted in your MySQL table. To verify this in your MySQL table, execute the following command from your MySQL client:

    SELECT * FROM employee;

    The following table rows are displayed:

    +-------------+---------------------+--------------------+------------------+| employee_id | employee_first_name | employee_last_name | employee_emailID |+-------------+---------------------+--------------------+------------------+|           1 | Peter               | Dilliard           | test-01@test.com ||           2 | Meaghan             | Webb               | test-02@test.com |+-------------+---------------------+--------------------+------------------+

What's next

• Insert data into BigQuery using a For Each Parallel task
• All triggers and tasks