Datasources & Dataformat

APICHAP differs datasources and dataformat. The concept of a datasource is the type of system where to get data from. The dataformat on the other hand is the format that data is received from a datasource.

For example: If you are calling an API (A Stateless GET request) to recieve a JSON entity, the Datasource is API and the Dataformat is JSON. In another case you might need to create an XML file on an FTP Server. Meaning your datasource is FTP-SERVER and the Dataformat is XML.

Most of our datasources types and dataformats can be mixed up to meet your needs.

Datasources

APICHAP is designed to work with multiple datasources. This means that you can get, write and manage data for multiple systems.

Type	Information
Database	Connect to a SQL database.
NoSQL Database	Connect to a No-SQL database.
API	Send and retrieve data via request to a 3rd party
FTP-Server	Read, Write and Manage files on an ftp server.

Roadmap: We are constantly expanding the list of supported datasources and datatypes. Connect us if you are missing one: Conteact.

Whenever you want to read, write or manage data in a fulfillment you must specify a datasource name. That datasource must be specified in the datasources tag. Depending on the type of your datasource it has different configuration options.

datasources:
  your-datasource-name:
    ....

Database

APICHAP currently supports the following databases:

mariadb / mysql
postgres
TSQL (Microsoft SQL, Azure SQL)

A database datasource has the following configuration options:

Field	Description
type	Specifies the type of datasource. Use the value `db` for database.	Mandatory
url	The JDBC URL for the datasource. In this case, the URL is "jdbc:mariadb://host.docker.internal:3306/sample-products".	Mandatory for `db`
user	The username for the datasource.	Optional, user can also be part of the url
password	The password for the datasource.	Optional, password can also be part of the url

datasources:
  demodb:
    type: db
    url: jdbc:mariadb://host.docker.internal:3306/sample-products
    user: testuser
    password: samplepwd

NoSQL-Database

Coming Soon

API

Specifying an API as a datasource. Right now an API datasource does not have any additional configuration options.

Field	Description
type	Specifies the type of datasource. Use the value `api` for APIs.	Mandatory

datasources:
  crm_api:
    type: api

FTP-Server

Specify the connection to a FTP-Server to read, write and move file.

Field	Description
type	Specifies the type of datasource. Use the value `ftp` for FTP-Server.	Mandatory
host	The hostname of the FTP-Server.	Mandatory
port	The port of your FTP-Server.	Optional
user	The user.	Mandatory
password	The password.	Mandatory

datasources:
  myftp:
    type: ftp
    host: The ftp hostname
    port: 21
    user: The ftp user
    password: ${ENV(FTP_PWD)}  # This uses a environment variable feature for sensitive data.

Since your API is shipped in a container we are only working with FTP passive-mode. If your FTP server blocks passive-mode the connection will not work.

Dataformat

The supported dataformats are:

Type	Content-Type
JSON	`application/json`
XML	`text/xml` or `application/xml`
CSV	`text/csv`
PLAIN-TEXT	`text/plain`

No matter which format, the configuration is always very similar.

Read: Whenever you receive data from one of the supported dataformats they are automatically transformed into a JSON format. Therefore you will always be able to receive data via the common VALUE() method using the JSONPath syntax.

Write: No matter which dataformat you want to write data in. The syntax will always work very similar, by creating what we call a schema. To learn how you can build a schema take a look at Schema Generation.

However, some of the dataformats have a few special rules on how to build a schema.

JSON

application/json

This schema:

Leads to the following JSON:

href="#__codelineno-4-1">schema: person: item: firstname: TEXT(Maria) lastname: TEXT(Lee) countries_visited: list: LIST_BY_VALUES(Austria,Japan,Kenya) item: name: VALUE(@, $) href="#__codelineno-5-1">{ "person": { "firstname": "Maria", "lastname": "Lee", "countries_visited": [ { "name": "Austria" }, { "name": "Japan" }, { "name": "Kenya" } ] } class="p">}

XML

text/xml
application/xml

Building an XML schema is similar to building a JSON schema, but there is one important note:

The rootElement is always the first key of the schema. This means that XML files must always start with a single root object.

Currently, XML attributes are not supported.

schema:
    person:
        item:
            firstname: TEXT(Maria)
            lastname: TEXT(Lee)
            countries_visited: 
                list: LIST_BY_VALUES(Austria,Japan,Kenya)
                item: 
                    name: VALUE(@, $)

Leads to the following XML:

<?xml version='1.0' encoding='UTF-8'?>
<person>
    <firstname>Maria</firstname>
    <lastname>Lee</lastname>
    <countries_visited>
        <name>Austria</name>
        <name>Japan</name>
        <name>Kenya</name>
    </countries_visited>
</person>

CSV

text/csv

By design CSV files can only have a single level of data, meaning it is not possible to create nested objects in them. The same applies for the CSV data you are building with APICHAP. It only allowes a single level of data variables.

The column seperator can be defined in the instruction. See details

schema:
    firstname: TEXT(Maria)
    lastname: TEXT(Lee)
    countries_visited_1: TEXT(Austria)
    countries_visited_2: TEXT(Japan)
    countries_visited_3: TEXT(Kenya)
    countries_visited_4: NULL()

Leads to the following CSV:

firstname;lastname;countries_visited_1;countries_visited_2;countries_visited_3;countries_visited_4;
Maria;Lee;Austria;Japan;Kenya;null;

Plain Text

text/plain

If you are only working with unknown or text data use the plain/text dataformat. This can be used to for example read a txt file uploaded to your server.

For writing plain data your schema is only allowed to have a single field: plain.

schema:
    plain: TEXT(Hello World)

Leads to the following txt file:

Hello World