There are three ways to ingest external data to chango.

• Ingest local json and excel data and json data located on S3 to chango using chango client.

• Ingest streaming json data to chango using chango client library.

• Ingest external data sources to chango using trino catalogs.

Internal Process in chango to ingest external data to chango

The following picture shows the internal process in chango to ingest json and excel to chango. Chango Data API Server will receive the incoming json and excel data, convert it to avro and forward to internal kafka. Internal spark streaming job will consume avro messages from kafka and insert them to iceberg table in chango s3 bucket. Simultaneously, all the data inserted into iceberg table can be queries by BI tools like Redash and Metabase provided by chango.

...