# Example Flow API Scripts

> **📝 Note**: This is a Preview feature.

## Example Bash Script: Automate Ingest Setup and Full Extract via API

This example Bash script automates the full setup of the Ingest UI checklist using API calls. It sets up a source MySQL database connection and a destination SingleStore database hosted in Helios, selects a table from a sample MySQL database called [classicmodels](https://www.mysqltutorial.org/getting-started-with-mysql/mysql-sample-database/), configures a periodic schedule, adds a license key, and finally triggers a full extract.

**Note**: Modify source and destination database configuration, table names, schedule, and license key to suit your environment.

* *View and Run the Bash Script*

  ```shell
  #!/bin/bash

  echo
  echo "Bash script to automate the Ingest UI checklist via API and trigger a full extract."
  echo "This script completes the following Ingest UI checklist steps:"
  echo "1. Setup Source"
  echo "2. Setup Destination"
  echo "3. Setup Tables"
  echo "4. Setup Schedule"
  echo "5. Enter Licence"

  BASE_URL="http://localhost:8081/ingest/api/ingest"
  HEADERS=(-H "Content-Type: application/json")

  verify_ui() {
      echo
      echo "Refresh the Flow UI and verify the following:"
      echo "$1"
      read -p "Press ENTER to continue..."
  }


  # -- Source Connection Setup --
  SRC_SUCCESS=false
  SRC_HOST="127.0.0.1" # Replace with your source database host name
  SRC_PORT="3306"      # Replace with your source database port number

  for attempt in {1..3}; do
      echo
      echo "Setting up source connection..."  
      read -p "Enter source database username: " SRC_UID
      read -rs -p "Enter source database password: " SRC_PWD

      SRC_PAYLOAD=$(jq -n --arg uid "$SRC_UID" --arg pwd "$SRC_PWD" --arg host "$SRC_HOST" --arg port "$SRC_PORT" '[ 
            {"name":"host","value":$host},                              
            {"name":"port","value":$port},
            {"name":"uid","value":$uid},
            {"name":"pwd","value":$pwd},
            {"name":"pwd2","value":$pwd},
            {"name":"type","value":"rds.mysqlc"}]'
      )
      
      curl -s -X PATCH "$BASE_URL/conn-src" "${HEADERS[@]}" -d "$SRC_PAYLOAD"
      echo
      verify_ui "Source connection should appear on the dashboard."

      echo
      echo "Testing source connection..."
      TEST_RESPONSE=$(curl -s -X PATCH "$BASE_URL/conn-src/test" "${HEADERS[@]}" -d "$SRC_PAYLOAD")

      if [[ $(echo "$TEST_RESPONSE" | jq -r '.success') == "true" ]]; then
          echo "Source connection test: SUCCESS."
          SRC_SUCCESS=true
          break
      else
          echo "FAILED. Try again: $(echo "$TEST_RESPONSE" | jq -r '.message')"
      fi
  done

  if [ "$SRC_SUCCESS" = false ]; then
      echo "Source connection setup failed after 3 attempts. Aborting."
      exit 1
  fi

  # -- Destination Connection Setup --
  DST_SUCCESS=false
  DST_HOST="dummy-host.singlestore.com" # Replace with your SingleStore host name
  DST_PORT="3306"  # Default port for SingleStore

  for attempt in {1..3}; do
      echo
      echo "Setting up destination connection..."
      read -p "Enter destination database username: " DST_UID
      read -rs -p "Enter destination database password: " DST_PWD

      DST_PAYLOAD=$(jq -n --arg uid "$DST_UID" --arg pwd "$DST_PWD" --arg host "$DST_HOST" --arg port "$DST_PORT" '[
            {"name":"host","value":$host},
            {"name":"port","value":$port},
            {"name":"uid","value":$uid},
            {"name":"pwd","value":$pwd},
            {"name":"pwd2","value":$pwd},
            {"name":"type","value":"rds.sstorem"}]'
      )
      
      curl -s -X PATCH "$BASE_URL/conn-dst" "${HEADERS[@]}" -d "$DST_PAYLOAD"
      echo
      verify_ui "Destination connection should appear on the dashboard."

      echo
      echo "Testing destination connection..."
      TEST_RESPONSE=$(curl -s -X PATCH "$BASE_URL/conn-dst/test" "${HEADERS[@]}" -d "$DST_PAYLOAD")
      if [[ $(echo "$TEST_RESPONSE" | jq -r '.success') == "true" ]]; then
          echo "Destination connection test: SUCCESS."
          DST_SUCCESS=true
          break
      else
          echo "FAILED. Try again: $(echo "$TEST_RESPONSE" | jq -r '.message')"
      fi
  done

  if [ "$DST_SUCCESS" = false ]; then
      echo "Destination connection setup failed after 3 attempts. Aborting."
      exit 1
  fi

  # -- Table Setup --
  # This script uses the table "customers" from the schema "classicmodels" and database "default", replace with your own configuration

  echo
  echo "Select a table for ingestion...."  

  TABLE_PAYLOAD='{
    "select": "Y"
  }'

  curl -s -X PATCH "$BASE_URL/config-tab/default/classicmodels/customers" "${HEADERS[@]}" -d "$TABLE_PAYLOAD"
  verify_ui "Verify that 'classicmodels.customers' is selected in the Tables tab."

  # -- Schedule Setup --

  SCHED_PAYLOAD='{
    "type": "periodic",
    "duration": "0d00h01m00s",
    "offset": "0d00h00m"
  }'

  curl -s -X PATCH "$BASE_URL/config-sched" "${HEADERS[@]}" -d "$SCHED_PAYLOAD"
  verify_ui "Schedule should show as periodic (every 1 minute) on the Scheduler tab."

  # -- License Key Setup --

  # Replace the dummy license key below with your actual license key.
  LIC_KEY="DUMMY-YYYML-ABCD-EFGH"

  LIC_PAYLOAD=$(jq -n --arg key "$LIC_KEY" '[
    {"name":"_lic_key","value":$key}]'
  )

  curl -s -X PATCH "$BASE_URL/config-var/licence" "${HEADERS[@]}" -d "$LIC_PAYLOAD"
  echo
  verify_ui "Check under 'Settings → Licence'. The license key should be valid and active."

  # -- Trigger Full Extract --
  echo
  if ! curl -s -X POST "$BASE_URL/ops/extract/full" "${HEADERS[@]}" | grep -q '"success":true'; then
      echo "Full extract failed."
      exit 1
  fi

  echo "Full extract triggered successfully."
  verify_ui "Check that a full extract job has started and is running and wait until completion. Also check 'Logs'."

  echo
  echo "Setup complete. You can now monitor the extract job in the Flow UI."

  ```## Run the ScriptEnsure the script has execute permissions by running the following command: ```shell
  chmod +x ingest_setup_and_full_extract.sh
  ```To run the script:```shell
  ./ingest_setup_and_full_extract.sh

  ```## Sample Output```shell
  $ ./ingest_setup_and_full_extract.sh

  Bash script to automate the Ingest UI checklist via API and trigger a full extract.
  This script completes the following Ingest UI checklist steps:
  Setup Source
  Setup Destination
  Setup Tables
  Setup Schedule
  Enter Licence

  Setting up source connection...
  Enter source database username: admin
  Enter source database password: 

  Refresh the Flow UI and verify the following:
  Source connection should appear on the dashboard.
  Press ENTER to continue...

  Testing source connection...
  Source connection test: SUCCESS.

  Setting up destination connection...
  Enter destination database username: admin
  Enter destination database password: 

  Refresh the Flow UI and verify the following:
  Destination connection should appear on the dashboard.
  Press ENTER to continue...

  Testing destination connection...
  Destination connection test: SUCCESS.

  Select a table for ingestion....

  Refresh the Flow UI and verify the following:
  Verify that 'classicmodels.customers' is selected in the Tables tab.
  Press ENTER to continue...

  Refresh the Flow UI and verify the following:
  Schedule should show as periodic (every 1 minute) on the Scheduler tab.
  Press ENTER to continue...

  Refresh the Flow UI and verify the following:
  Check under 'Settings → Licence'. The license key should be valid and active.
  Press ENTER to continue...

  Full extract triggered successfully.

  Refresh the Flow UI and verify the following:
  Check that a full extract job has started and is running and wait until completion. Also check 'Logs'.
  Press ENTER to continue...

  Setup complete. You can now monitor the extract job in the Flow UI.
  ```## Verify Data in the Destination Database```sql
  SELECT COUNT(*) FROM customers;
  +----------------------------------+
  | COUNT(*)                         |
  | 122                              |
  +----------------------------------+

  ```

## Example Python Script: Automate Ingest Setup and Full Extract via API

This example Python script automates the full setup of the Ingest UI checklist using API calls. It sets up a source MySQL database connection and a destination SingleStore database hosted in Helios, selects a table from a sample MySQL database called [classicmodels](https://www.mysqltutorial.org/getting-started-with-mysql/mysql-sample-database/), configures a periodic schedule, adds a license key, and finally triggers a full extract.

**Note**: Modify source and destination database configuration, tables, schedule, and license key to suit your environment.

* *View and Run the Python Script*

  ```python3
  import warnings
  warnings.filterwarnings("ignore")

  import requests
  import pwinput
  import time
  import sys

  def verify_ui(instruction):
      print()
      print("Refresh the Flow UI and verify the following:")
      print(instruction)
      input("Press ENTER to continue...")

  print()
  print ("Python script to automate the Ingest UI checklist via API and trigger a full extract.") 
  print ("This script completes the following Ingest UI checklist steps:")
  print ("Setup Source")
  print ("Setup Destination")
  print ("Setup Tables")
  print ("Setup Schedule")
  print ("Enter Licence")

  time.sleep(1)

  # Set up source connection
  SRC_HOST = "127.0.0.1"   # Replace with your source database host name
  SRC_PORT = "3306"        # Replace with your source database port number

  for attempt in range(3):
      try:
          print()
          print("Setting up source connection...")
          src_uid = input("Enter source database username: ")
          # Password prompt works cross-platform (Mac/Windows/Linux) except on Windows PowerShell, where you cannot paste the password with Ctrl-V. 
          # Use right click or the PowerShell paste menu instead.
          src_pwd = pwinput.pwinput("Enter source database password: ")

          src_payload = [
              {"name": "host", "value": SRC_HOST}, 
              {"name": "port", "value": SRC_PORT},
              {"name": "uid", "value": src_uid},
              {"name": "pwd", "value": src_pwd},
              {"name": "pwd2", "value": src_pwd},
              {"name": "type", "value": "rds.mysqlc"},
              {"name": "db", "value": ""}
          ]
          
          r = requests.patch("http://localhost:8081/ingest/api/ingest/conn-src",
                              json=src_payload,
                              headers={"Content-Type": "application/json"})
          r.raise_for_status()
          verify_ui("Source connection should appear on the dashboard.")

          # Test source connection
          print()
          print("Testing source connection...")
          time.sleep(1)
          
          r = requests.patch(
              "http://localhost:8081/ingest/api/ingest/conn-src/test",
              json=src_payload,
              headers={"Content-Type": "application/json"}
          )
          r.raise_for_status()
          resp_json = r.json()
              
          if resp_json.get("success") is True:
              print("Source connection test: SUCCESS. You may proceed to the next step.")
              break
          else:
              error_message = resp_json.get("message")
              print("Source connection test: FAILED. Please verify your credentials and try again.", error_message)
      except Exception as e:
              print("Error communicating with API:", e)

      if attempt == 2:
          print("Failed to connect to source after 3 attempts. Exiting.")
          sys.exit(1)
      else:
          print()
          print("Please verify credentials and try again.\n")

  # Set up destination connection
  DST_HOST = "dummy-host.singlestore.com" # Replace with your SingleStore host name
  DST_PORT="3306"  # Default port for SingleStore


  for attempt in range(3):
      try:
          print()
          print("Setting up destination connection...")
          dst_uid = input("Enter destination database username: ")
          # See earlier comment about Ctrl-V issue when using Windows PowerShell with pwinput.
          dst_pwd = pwinput.pwinput("Enter destination database password: ")

          dst_payload = [
              {"name": "host", "value": DST_HOST},
              {"name": "port", "value": DST_PORT},
              {"name": "uid", "value": dst_uid},
              {"name": "pwd", "value": dst_pwd},
              {"name": "pwd2", "value": dst_pwd},
              {"name": "type", "value": "rds.sstorem"},
              {"name": "db", "value": ""}
          ]
          r = requests.patch("http://localhost:8081/ingest/api/ingest/conn-dst",
                              json=dst_payload,
                              headers={"Content-Type": "application/json"})
          r.raise_for_status()
          verify_ui("Destination connection should appear on the dashboard.")

          # Test destination connection
          print()
          print("Testing destination connection...")
          time.sleep(1)

          r = requests.patch(
              "http://localhost:8081/ingest/api/ingest/conn-dst/test",
              json=dst_payload,
              headers={"Content-Type": "application/json"}
          )
          r.raise_for_status()
          resp_json = r.json()

          if resp_json.get("success") is True:
              print("Destination connection test: SUCCESS. You may proceed to the next step.")
              break
          else:
              error_message = resp_json.get("message")
              print("Destination connection test: FAILED. Please verify your credentials and try again.", error_message)
      except Exception as e:
          print("Error communicating with API:", e)

      if attempt == 2:
          print("Failed to connect to destination after 3 attempts. Exiting.")
          sys.exit(1)
      else:
          print()
          print("Please verify credentials and try again.\n")


  # Select table
  # This script uses the table "customers" from the schema "classicmodels" and database "default", replace with your own configuration

  print()
  print("Configuring table classicmodels.customers...")

  tab_payload = {
      "select": "Y"
  }

  r = requests.patch("http://localhost:8081/ingest/api/ingest/config-tab/default/classicmodels/customers",
                      json=tab_payload,
                      headers={"Content-Type": "application/json"})
  r.raise_for_status()
  verify_ui("Verify that 'classicmodels.customers' is selected in the Tables tab.")


  # Set up schedule

  print()
  print("Setting up the scheduler...")

  sched_payload = {
  "type": "periodic",
  "duration": "0d00h01m00s",
  "offset": "0d00h00m"
  }

  r = requests.patch("http://localhost:8081/ingest/api/ingest/config-sched",
                      json=sched_payload,
                      headers={"Content-Type": "application/json"})
  r.raise_for_status()
  verify_ui("Schedule should show as periodic (every 1 minute) on the Scheduler tab.")

  # Set up License

  print()
  LIC_KEY = "DUMMY-YYYML-ABCD-EFGH" # Dummy license key, replace with your valid license key.

  lic_payload = [
      {"name": "_lic_key", "value": LIC_KEY}
  ]

  r = requests.patch("http://localhost:8081/ingest/api/ingest/config-var/licence",
                      json=lic_payload,
                      headers={"Content-Type": "application/json"})
  r.raise_for_status()
  verify_ui("Check under 'Settings → Licence'. The license key should be valid and active.")

  # Trigger full extract

  print()
  print("Triggering full extract...")

  r = requests.post("http://localhost:8081/ingest/api/ingest/ops/extract/full",
                      headers={"Content-Type": "application/json"})
  r.raise_for_status()

  if '"success":true' not in r.text:
      print("Full extract failed.")
      exit(1)

  verify_ui("Check that a full extract job has started and is running and wait until completion. Also check 'Logs'.")

  print("\nSetup complete. You can now monitor the extract job in the Flow UI.")

  ```## Run the Script**Prerequisites**- Python 3.6+ is installed
  - The following packages are installed:
    ```shell
    pip install requests
    pip install pwinput
    ```
  - Optional: Use a virtual environment to isolate dependencies
    ```shell
    python3 -m venv venvm
    source venvm/bin/activate    # On Windows: venvm\Scripts\activate
    pip install requests
    ```To run the script:```shell
  $ python3 ingest_setup_and_full_extract.py

  ```## Sample Output```shell
  $ python3 ingest_setup_and_full_extract.py

  Python script to automate the Ingest UI checklist via API and trigger a full extract.
  This script completes the following Ingest UI checklist steps:
  Setup Source
  Setup Destination
  Setup Tables
  Setup Schedule
  Enter Licence

  Setting up source connection...
  Enter source database username: root
  Enter source database password: 

  Refresh the Flow UI and verify the following:
  Source connection should appear on the dashboard.
  Press ENTER to continue...

  Testing source connection...
  Source connection test: SUCCESS. You may proceed to the next step.

  Setting up destination connection...
  Enter destination database username: admin
  Enter destination database password: 

  Refresh the Flow UI and verify the following:
  Destination connection should appear on the dashboard.
  Press ENTER to continue...

  Testing destination connection...
  Destination connection test: SUCCESS. You may proceed to the next step.

  Configuring table classicmodels.customers...

  Refresh the Flow UI and verify the following:
  Verify that 'classicmodels.customers' is selected in the Tables tab.
  Press ENTER to continue...

  Setting up the scheduler...

  Refresh the Flow UI and verify the following:
  Schedule should show as periodic (every 1 minute) on the Scheduler tab.
  Press ENTER to continue...


  Refresh the Flow UI and verify the following:
  Check under 'Settings → Licence'. The license key should be valid and active.
  Press ENTER to continue...

  Triggering full extract...

  Refresh the Flow UI and verify the following:
  Check that a full extract job has started and is running and wait until completion. Also check 'Logs'.
  Press ENTER to continue...

  Setup complete. You can now monitor the extract job in the Flow UI.

  ```## Verify Data in the Destination Database```sql
  SELECT COUNT(*) FROM customers;
  +----------------------------------+
  | COUNT(*)                         |
  | 122                              |
  +----------------------------------+

  ```

## Example Bash Script: Automate Sliced Data Transfer via XL Ingest APIs

This example Bash script automates sliced data transfer using XL Ingest APIs. It assumes that the source and destination database connections are already configured in Ingest. The script selects a table, configures it to skip the initial extract (schema-only transfer), triggers a full extract, applies a license key in XL Ingest, sets up table slices based on a column (orderNumber), and finally triggers a sliced data transfer.

**Note**: Modify table names, slice values, and license key to suit your environment.

* *View and Run the Bash Script*

  ```shell
  #!/bin/bash

  echo
  echo "Bash script to automate sliced data transfer using XL Ingest. "
  echo "Ensure you have source and destination databases connected in Ingest prior to running this script."
  echo "Steps:"
  echo "1. Skip Initial Extract (schema-only) in Ingest"
  echo "2. Trigger Full Extract in Ingest"
  echo "3. Apply License Key in XL Ingest"
  echo "4. Configure Table Slices"
  echo "5. Trigger Sliced Transfer"
  echo

  # -- Config --
  BASE_URL_INGEST="http://localhost:8081/ingest/api/ingest"
  BASE_URL_XL_INGEST="http://localhost:8084/ixl"
  HEADERS=(-H "Content-Type: application/json")

  verify_ui() {
      echo
      echo "Please verify in UI: $1"
      read -p "Press ENTER to continue..."
  }


  # -- Select the table to be ingested in Ingest and enable Skip Initial Extract --

  echo "Select the table and skip initial extract (schema-only copy)..."
  SKIP_PAYLOAD='{
    "select": "Y",
    "skip": "Y"
  }'

  curl -s -X PATCH "$BASE_URL_INGEST/config-tab/default/classicmodels/orderdetails" "${HEADERS[@]}" -d "$SKIP_PAYLOAD"
  verify_ui "'classicmodels.orderdetails' should be selected in Ingest and set to skip extract (schema-only)."

  # -- Trigger Full Extract and transfer only the schema to destination --

  echo
  echo "Trigger full extract for the selected table..."
  if ! curl -s -X POST "$BASE_URL_INGEST/ops/extract/full" "${HEADERS[@]}" | grep -q '"success":true'; then
      echo "Full extract failed."
      exit 1
  fi

  echo "Full extract triggered successfully."
  verify_ui "Check that a full extract job has started and is running and wait until completion. Also check Logs."

  # -- License Key Setup in XL Ingest--

  # Replace the dummy license key below with your actual license key.

  echo "Set up the license key for XL Ingest.."
  LIC_KEY="BTLQ2-SSYML-J6TFT-UXKZT-RLHPZ-HZAIH"

  LIC_PAYLOAD=$(jq -n --arg key "$LIC_KEY" '[
    {"name":"key","value":$key}]'
  )

  curl -s -X PATCH "$BASE_URL_XL_INGEST?cmd=cfp" "${HEADERS[@]}" -d "$LIC_PAYLOAD"
  echo
  verify_ui "Check under 'Configuration'. The license key should be valid and active."

  # -- Configure Table Slices in XL Ingest --

  echo
  echo "Configure table slices via XL Ingest..."

  SLICE_PAYLOAD='{
    "id": "default:classicmodels.orderdetails",
    "sliceCol": "orderNumber",
    "sliceList": [
      {"name": "10099"},
      {"name": "10200"},
      {"name": "10300"},
      {"name": "10400"},
      {"name": "10500"}
    ],
    "select": true
  }'

  SLICE_RESPONSE=$(curl -s -X PATCH "$BASE_URL_XL_INGEST?cmd=tdp" "${HEADERS[@]}" -d "$SLICE_PAYLOAD")
  SLICE_SUCCESS=$(echo "$SLICE_RESPONSE" | jq -r '.success')

  if [[ "$SLICE_SUCCESS" != "true" ]]; then
      echo "Slice configuration FAILED: $(echo "$SLICE_RESPONSE" | jq -r '.message // .error // "Unknown error"')"
      exit 1
  fi

  echo "Slice configuration SUCCESS."
  verify_ui "Check that table 'orderdetails' is sliced correctly in XL Ingest."

  # -- Trigger Sliced Transfer --

  echo
  echo "Trigger sliced data transfer..."

  curl -s -X POST "$BASE_URL_XL_INGEST?cmd=tgsy&tab=default%3Aclassicmodels.orderdetails" "${HEADERS[@]}"
  echo "Sliced transfer triggered."
  verify_ui "Check that the sliced data transfer job has started and is progressing/completed in the UI."
  ```## Run the ScriptEnsure the script has execute permissions by running the following command: ```shell
  chmod +x xl_ingest_sliced_transfer.sh
  ```To run the script:```shell
  ./xl_ingest_sliced_transfer.sh

  ```## Sample Output```shell
  $ ./xl_ingest_sliced_transfer.sh

  Bash script to automate sliced data transfer using XL Ingest. 
  Ensure you have source and destination databases connected in Ingest prior to running this script.
  Steps:
  1. Skip Initial Extract (schema-only) in Ingest
  2. Trigger Full Extract in Ingest
  3. Apply License Key in XL Ingest
  4. Configure Table Slices
  5. Trigger Sliced Transfer

  Select the table and skip initial extract (schema-only copy)...

  Please verify in UI: 'classicmodels.orderdetails' should be selected and set to skip extract (schema-only).
  Press ENTER to continue...

  Trigger full extract for the selected table...
  Full extract triggered successfully.

  Please verify in UI: Check that a full extract job has started and is running and wait until completion. Also check Logs.
  Press ENTER to continue...
  Set up the license key for XL Ingest..
  {"success":true}


  Please verify in UI: Check under 'Configuration'. The license key should be valid and active.
  Press ENTER to continue...

  Configure table slices via XL Ingest...
  Slice configuration SUCCESS.

  Please verify in UI: Check that table 'orderdetails' has correct slice configuration in XL Ingest.
  Press ENTER to continue...

  Trigger sliced data transfer...
  {"isRunning":true,"type":"SY","tasksComplete":0,"table":"default:classicmodels.orderdetails","tasksTotal":4}
  Sliced transfer triggered.

  Please verify in UI: Check that the sliced data transfer job has started and is progressing/completed in the UI.

  ```## Verify Data in the Destination Database```sql
  SELECT * FROM orderdetails LIMIT 5;
  +------------------------------------------------------------------+
  |orderNumber productCode quantityOrdered priceEach orderLineNumber |
  |10225       S18_3278    37              69.96     12              | 
  |10174       S18_1097    48              108.50    3               | 
  |10235       S700_3167   32              73.60     9               |
  |10148       S24_3432    27              96.37     7               |
  |10211       S12_3990    28              79.80     3               |         
  +------------------------------------------------------------------+
  ```

## Example Python Script: Automate Sliced Data Transfer via XL Ingest APIs

This example Python script automates sliced data transfer using XL Ingest APIs. It assumes that the source and destination database connections are already configured in Ingest. The script selects a table, configures it to skip the initial extract (schema-only transfer), triggers a full extract, applies a license key in XL Ingest, sets up table slices based on a column (orderNumber), and finally triggers a sliced data transfer.

**Note**: Modify table names, slice values, and license key to suit your environment.

* *View and Run the Python Script*

  ```python3
  import warnings
  warnings.filterwarnings("ignore")

  import requests
  import time
  import sys

  def verify_ui(instruction):
      print()
      print("Refresh the Flow UI and verify the following:")
      print(instruction)
      input("Press ENTER to continue...")

  print()
  print("Python script to automate sliced data transfer using XL Ingest.")
  print("Ensure you have source and destination databases connected in Ingest prior to running this script.")
  print("Steps:")
  print("1. Skip Initial Extract (schema-only) in Ingest")
  print("2. Trigger Full Extract in Ingest")
  print("3. Apply License Key in XL Ingest")
  print("4. Configure Table Slices")
  print("5. Trigger Sliced Transfer")

  time.sleep(1)

  # -- Config --
  BASE_URL_INGEST = "http://localhost:8081/ingest/api/ingest"
  BASE_URL_XL_INGEST = "http://localhost:8084/ixl"
  HEADERS = {"Content-Type": "application/json"}

  # -- Select the table and skip initial extract (schema-only copy) --
  print()
  print("Select the table and skip initial extract (schema-only copy)...")

  skip_payload = {
      "select": "Y",
      "skip": "Y"
  }

  r = requests.patch(f"{BASE_URL_INGEST}/config-tab/default/classicmodels/orderdetails",
                     json=skip_payload,
                     headers=HEADERS)
  r.raise_for_status()
  verify_ui("'classicmodels.orderdetails' should be selected in Ingest and set to skip extract (schema-only).")

  # -- Trigger Full Extract and transfer only the schema --
  print()
  print("Trigger full extract for the selected table...")

  r = requests.post(f"{BASE_URL_INGEST}/ops/extract/full", headers=HEADERS)
  r.raise_for_status()

  if '"success":true' not in r.text:
      print("Full extract failed.")
      sys.exit(1)

  print("Full extract triggered successfully.")
  verify_ui("Check that a full extract job has started and is running or completed. Also check Logs.")

  # -- License Key Setup in XL Ingest --
  print()
  print("Set up the license key for XL Ingest...")

  LIC_KEY = "BTLQ2-SSYML-J6TFT-UXKZT-RLHPZ-HZAIH"  # Dummy license key, replace with your actual key

  lic_payload = [
      {"name": "key", "value": LIC_KEY}
  ]

  r = requests.patch(f"{BASE_URL_XL_INGEST}?cmd=cfp", json=lic_payload, headers=HEADERS)
  r.raise_for_status()
  verify_ui("Check under 'Configuration'. The license key should be valid and active.")

  # -- Configure Table Slices in XL Ingest --
  print()
  print("Configure table slices via XL Ingest...")

  slice_payload = {
      "id": "default:classicmodels.orderdetails",
      "sliceCol": "orderNumber",
      "sliceList": [
          {"name": "10099"},
          {"name": "10200"},
          {"name": "10300"},
          {"name": "10400"},
          {"name": "10500"}
      ],
      "select": True
  }

  r = requests.patch(f"{BASE_URL_XL_INGEST}?cmd=tdp", json=slice_payload, headers=HEADERS)
  r.raise_for_status()

  resp_json = r.json()
  if not resp_json.get("success"):
      print("Slice configuration FAILED:", resp_json.get("message") or resp_json.get("error") or "Unknown error")
      sys.exit(1)

  print("Slice configuration SUCCESS.")
  verify_ui("Check that table 'orderdetails' has correct slice configuration in XL Ingest.")

  # -- Trigger Sliced Transfer --
  print()
  print("Trigger sliced data transfer...")

  r = requests.post(f"{BASE_URL_XL_INGEST}?cmd=tgsy&tab=default%3Aclassicmodels.orderdetails", headers=HEADERS)
  r.raise_for_status()
  print("Sliced transfer triggered.")
  print("Check that the sliced data transfer job has started and is progressing/completed in the UI.")
  ```## Run the Script**Prerequisites**- Python 3.6+ is installed
  - The following packages are installed:
    ```shell
    pip install requests

    ```
  - Optional: Use a virtual environment to isolate dependencies
    ```shell
    python3 -m venv venvm
    source venvm/bin/activate    # On Windows: venvm\Scripts\activate
    pip install requests
    ```To run the script:```shell
  $ python3 xl_ingest_sliced_transfer.py

  ```## Sample Output```shell
  $ python3 xl_ingest_sliced_transfer.py

  Python script to automate sliced data transfer using XL Ingest.
  Ensure you have source and destination databases connected in Ingest prior to running this script.
  Steps:
  1. Skip Initial Extract (schema-only) in Ingest
  2. Trigger Full Extract in Ingest
  3. Apply License Key in XL Ingest
  4. Configure Table Slices
  5. Trigger Sliced Transfer

  Select the table and skip initial extract (schema-only copy)...

  Refresh the Flow UI and verify the following:
  'classicmodels.orderdetails' should be selected and set to skip extract (schema-only).
  Press ENTER to continue...

  Trigger full extract for the selected table...
  Full extract triggered successfully.

  Refresh the Flow UI and verify the following:
  Check that a full extract job has started and is running or completed. Also check Logs.
  Press ENTER to continue...

  Set up the license key for XL Ingest...

  Refresh the Flow UI and verify the following:
  Check under 'Configuration'. The license key should be valid and active.
  Press ENTER to continue...

  Configure table slices via XL Ingest...
  Slice configuration SUCCESS.

  Refresh the Flow UI and verify the following:
  Check that table 'orderdetails' is sliced correctly in XL Ingest.
  Press ENTER to continue...

  Trigger sliced data transfer...
  Sliced transfer triggered.
  Check that the sliced data transfer job has started and is progressing/completed in the UI.

  ```## Verify Data in the Destination Database```sql
  SELECT * FROM orderdetails LIMIT 5;
  +------------------------------------------------------------------+
  |orderNumber productCode quantityOrdered priceEach orderLineNumber |
  |10225       S18_3278    37              69.96     12              | 
  |10174       S18_1097    48              108.50    3               | 
  |10235       S700_3167   32              73.60     9               |
  |10148       S24_3432    27              96.37     7               |
  |10211       S12_3990    28              79.80     3               |         
  +------------------------------------------------------------------+
  ```

***

Modified at: October 17, 2025

Source: [/db/v9.1/load-data/load-data-with-singlestore-flow/singlestore-flow-api/example-flow-api-scripts/](https://docs.singlestore.com/db/v9.1/load-data/load-data-with-singlestore-flow/singlestore-flow-api/example-flow-api-scripts/)

(An index of the documentation is available at /llms.txt)