ABEJA Dataset documentation

ABEJA dataset library is (SDK) for python, which allow developers to create, get and delete dataset and dataset items.

High level API

High level APIs are used trough following classes.

• Client
  • Client
    • Client.datasets
    • Client.get_dataset()
  • Dataset
    • Dataset
      • Dataset.dataset_items
  • Datasets
    • Datasets
      • Datasets.create()
      • Datasets.delete()
      • Datasets.get()
      • Datasets.list()
  • Dataset Item
    • DatasetItem
      • DatasetItem.asdict()
  • Dataset Items
    • DatasetItems
      • DatasetItems.bulk_update()
      • DatasetItems.create()
      • DatasetItems.delete()
      • DatasetItems.get()
      • DatasetItems.list()
      • DatasetItems.update()

Usage

from abeja.datasets import Client

client = Client(organization_id)
dataset = client.get_dataset(dataset_id)
for item in dataset.dataset_items.list():
   pass

API Mapping

method	path
post	/organizations/<organization_id>/datasets	Datasets.create()
get	/organizations/<organization_id>/datasets	Datasets.list()
get	/organizations/<organization_id>/datasets/<id>	Datasets.get()
delete	/organizations/<organization_id>/datasets/<id>	Datasets.delete()
post	/organizations/<organization_id>/datasets/<dataset_id>/items	DatasetItems.create()
get	/organizations/<organization_id>/datasets/<dataset_id>/items	DatasetItems.list()
get	/organizations/<organization_id>/datasets/<dataset_id>/items/<id>	DatasetItems.get()
update	/organizations/<organization_id>/datasets/<dataset_id>/items/<id>	DatasetItems.update()
bulk_update	/organizations/<organization_id>/datasets/<dataset_id>/items	DatasetItems.bulk_update()
delete	/organizations/<organization_id>/datasets/<dataset_id>/items/<id>	DatasetItems.delete()

Low level API

Low lavel apis are directly access the api endpoint.

• API Client
  • APIClient
    • APIClient.bulk_update_dataset_item()
    • APIClient.create_dataset()
    • APIClient.create_dataset_item()
    • APIClient.delete_dataset()
    • APIClient.delete_dataset_item()
    • APIClient.get_dataset()
    • APIClient.get_dataset_item()
    • APIClient.list_dataset_items()
    • APIClient.list_datasets()
    • APIClient.update_dataset_item()

Usage

from abeja.datasets import APIClient

api = APIClient()
dataset = api.get_dataset(organization_id, dataset_id)
dataset_items = api.list_dataset_items(organization_id, dataset_id)

API Mapping

method	path
post	/organizations/<organization_id>/datasets	APIClient.create_dataset()
get	/organizations/<organization_id>/datasets	APIClient.list_datasets()
get	/organizations/<organization_id>/datasets/<id>	APIClient.get_dataset()
delete	/organizations/<organization_id>/datasets/<id>	APIClient.delete_dataset()
post	/organizations/<organization_id>/datasets/<dataset_id>/items	APIClient.create_dataset_item()
get	/organizations/<organization_id>/datasets/<dataset_id>/items	APIClient.list_dataset_items()
get	/organizations/<organization_id>/datasets/<dataset_id>/items/<id>	APIClient.get_dataset_item()
update	/organizations/<organization_id>/datasets/<dataset_id>/items/<id>	APIClient.update_dataset_item()
bulk_update	/organizations/<organization_id>/datasets/<dataset_id>/items	APIClient.bulk_update_dataset_item()
delete	/organizations/<organization_id>/datasets/<dataset_id>/items/<id>	APIClient.delete_dataset_item()

Tutorial

• A Sample tutorial
  • Register dataset and dataset items
    • Steps 1: Upload data item to datalake
    • Step 2-1: Create Classification Dataset and Dataset Items
    • Step 2-2: Create Detection Dataset and Dataset Items
  • Use datasets