{ "cells": [ { "cell_type": "markdown", "id": "92475c69", "metadata": {}, "source": [ "# Python API Example - LNG Hubs (Cargo) API\n", "## Calling the latest & historical Hubs FOB & DES posts\n", "\n", "Here we call currently Active & Inactive (i.e. Historical) LNG Hubs FOB & DES posts for all available regions\n", "\n", "### Have any questions?\n", "\n", "If you have any questions regarding our API, or need help accessing specific datasets, please contact us at:\n", "\n", "__data@sparkcommodities.com__\n", "\n", "or refer to our API website for more information about this endpoint:\n", "https://www.sparkcommodities.com/api/" ] }, { "cell_type": "markdown", "id": "c5716130", "metadata": {}, "source": [ "## 1. Importing Data\n", "\n", "Here we define the functions that allow us to retrieve the valid credentials to access the Spark API.\n", "\n", "__This section can remain unchanged for most Spark API users.__" ] }, { "cell_type": "code", "execution_count": null, "id": "33fb0640", "metadata": {}, "outputs": [], "source": [ "import json\n", "import os\n", "import sys\n", "import pandas as pd\n", "import numpy as np\n", "from base64 import b64encode\n", "from pprint import pprint\n", "from urllib.parse import urljoin\n", "from datetime import datetime\n", "\n", "\n", "try:\n", " from urllib import request, parse\n", " from urllib.error import HTTPError\n", "except ImportError:\n", " raise RuntimeError(\"Python 3 required\")\n", "\n", "\n", "API_BASE_URL = \"https://api.sparkcommodities.com\"\n", "\n", "\n", "def retrieve_credentials(file_path=None):\n", " \"\"\"\n", " Find credentials either by reading the client_credentials file or reading\n", " environment variables\n", " \"\"\"\n", " if file_path is None:\n", " client_id = os.getenv(\"SPARK_CLIENT_ID\")\n", " client_secret = os.getenv(\"SPARK_CLIENT_SECRET\")\n", " if not client_id or not client_secret:\n", " raise RuntimeError(\n", " \"SPARK_CLIENT_ID and SPARK_CLIENT_SECRET environment vars required\"\n", " )\n", " else:\n", " # Parse the file\n", " if not os.path.isfile(file_path):\n", " raise RuntimeError(\"The file {} doesn't exist\".format(file_path))\n", "\n", " with open(file_path) as fp:\n", " lines = [l.replace(\"\\n\", \"\") for l in fp.readlines()]\n", "\n", " if lines[0] in (\"clientId,clientSecret\", \"client_id,client_secret\"):\n", " client_id, client_secret = lines[1].split(\",\")\n", " else:\n", " print(\"First line read: '{}'\".format(lines[0]))\n", " raise RuntimeError(\n", " \"The specified file {} doesn't look like to be a Spark API client \"\n", " \"credentials file\".format(file_path)\n", " )\n", "\n", " print(\">>>> Found credentials!\")\n", " print(\n", " \">>>> Client_id={}, client_secret={}****\".format(client_id, client_secret[:5])\n", " )\n", "\n", " return client_id, client_secret\n", "\n", "\n", "def do_api_post_query(uri, body, headers):\n", " url = urljoin(API_BASE_URL, uri)\n", "\n", " data = json.dumps(body).encode(\"utf-8\")\n", "\n", " # HTTP POST request\n", " req = request.Request(url, data=data, headers=headers)\n", " try:\n", " response = request.urlopen(req)\n", " except HTTPError as e:\n", " print(\"HTTP Error: \", e.code)\n", " print(e.read())\n", " sys.exit(1)\n", "\n", " resp_content = response.read()\n", "\n", " # The server must return HTTP 201. Raise an error if this is not the case\n", " assert response.status == 201, resp_content\n", "\n", " # The server returned a JSON response\n", " content = json.loads(resp_content)\n", "\n", " return content\n", "\n", "\n", "def do_api_get_query(uri, access_token, format='json'):\n", " \"\"\"\n", " After receiving an Access Token, we can request information from the API.\n", " \"\"\"\n", " url = urljoin(API_BASE_URL, uri)\n", "\n", " if format == 'json':\n", " headers = {\n", " \"Authorization\": \"Bearer {}\".format(access_token),\n", " \"Accept\": \"application/json\",\n", " }\n", " elif format == 'csv':\n", " headers = {\n", " \"Authorization\": \"Bearer {}\".format(access_token),\n", " \"Accept\": \"text/csv\"\n", " }\n", " else:\n", " raise AttributeError('The format parameter only takes `csv` or `json` as inputs')\n", "\n", " # HTTP GET request\n", " req = request.Request(url, headers=headers)\n", " try:\n", " response = request.urlopen(req)\n", " except HTTPError as e:\n", " print(\"HTTP Error: \", e.code)\n", " print(e.read())\n", " sys.exit(1)\n", "\n", " resp_content = response.read()\n", " #status = response.status\n", "\n", " # The server must return HTTP 200. Raise an error if this is not the case\n", " assert response.status == 200, resp_content\n", "\n", " # Storing response based on requested format\n", " if format == 'json':\n", " content = json.loads(resp_content)\n", " elif format == 'csv':\n", " content = resp_content\n", " \n", "\n", " return content\n", "\n", "\n", "def get_access_token(client_id, client_secret):\n", " \"\"\"\n", " Get a new access_token. Access tokens are the thing that applications use to make\n", " API requests. Access tokens must be kept confidential in storage.\n", "\n", " # Procedure:\n", "\n", " Do a POST query with `grantType` in the body. A basic authorization\n", " HTTP header is required. The \"Basic\" HTTP authentication scheme is defined in\n", " RFC 7617, which transmits credentials as `clientId:clientSecret` pairs, encoded\n", " using base64.\n", " \"\"\"\n", "\n", " # Note: for the sake of this example, we choose to use the Python urllib from the\n", " # standard lib. One should consider using https://requests.readthedocs.io/\n", "\n", " payload = \"{}:{}\".format(client_id, client_secret).encode()\n", " headers = {\n", " \"Authorization\": \"Basic {}\".format(b64encode(payload).decode()),\n", " \"Accept\": \"application/json\",\n", " \"Content-Type\": \"application/json\",\n", " }\n", " body = {\n", " \"grantType\": \"clientCredentials\",\n", " }\n", "\n", " content = do_api_post_query(uri=\"/oauth/token/\", body=body, headers=headers)\n", "\n", " print(\n", " \">>>> Successfully fetched an access token {}****, valid {} seconds.\".format(\n", " content[\"accessToken\"][:5], content[\"expiresIn\"]\n", " )\n", " )\n", "\n", " return content[\"accessToken\"]\n", "\n", "\n" ] }, { "cell_type": "markdown", "id": "fd3171a8", "metadata": {}, "source": [ "## N.B. Credentials\n", "\n", "Here we call the above functions, and input the file path to our credentials.\n", "\n", "N.B. You must have downloaded your client credentials CSV file before proceeding. You can create and download API credentials from the Spark Platform:\n", "\n", "https://app.sparkcommodities.com/data-integrations/api\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "id": "fd7e89bf", "metadata": {}, "outputs": [], "source": [ "# Insert file path to your client credentials here\n", "client_id, client_secret = retrieve_credentials(file_path=\"/tmp/client_credentials.csv\")\n", "\n", "# Authenticate:\n", "access_token = get_access_token(client_id, client_secret)\n", "print(access_token)" ] }, { "cell_type": "markdown", "id": "8fa4f6cd", "metadata": {}, "source": [ "# 2. Reference Data\n", "\n", "Here we fetch the relevant reference data for the endpoint. This includes region and incoterm codes that can be used in the data-fetching functions later in the script" ] }, { "cell_type": "code", "execution_count": null, "id": "5e02319b", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "v1.0/lng/hubs/cargo/reference-data/\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
RegionIncotermSlotType
0usgfobslot-swap
1usgfoboutright
2waffobslot-swap
3waffoboutright
4megfobslot-swap
\n", "
" ], "text/plain": [ " Region Incoterm SlotType\n", "0 usg fob slot-swap\n", "1 usg fob outright\n", "2 waf fob slot-swap\n", "3 waf fob outright\n", "4 meg fob slot-swap" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from io import StringIO\n", "\n", "## Defining the function\n", "\n", "def fetch_ref_data(access_token, format='json'):\n", "\n", " uri=\"v1.0/lng/hubs/cargo/reference-data/\"\n", " print(uri)\n", " \n", " content = do_api_get_query(\n", " uri=uri, access_token=access_token, format=format\n", " )\n", " \n", " if format == 'json':\n", " my_dict = content\n", " elif format == 'csv':\n", " # if there's no data to show, returns raw response (empty string) and \"No Data to Show\" message\n", " if len(content) == 0:\n", " my_dict = content\n", " print('No Data to Show')\n", " else:\n", " my_dict = content.decode('utf-8')\n", " my_dict = pd.read_csv(StringIO(my_dict)) # automatically converting into a Pandas DataFrame when choosing CSV format\n", " \n", " return my_dict\n", "\n", "ref_df = fetch_ref_data(access_token, format='csv')\n", "ref_df.head(5)" ] }, { "cell_type": "markdown", "id": "0994ce16", "metadata": {}, "source": [ "# 3. Active Posts\n", "\n", "Here we define the function used to call the currently live Hubs Cargo posts. This function takes three parameters:\n", "\n", "- 'incoterm': define which incoterm you'd like to filter by, 'fob' or 'des'\n", "\n", "- 'region': choose which region you'd like to filter by. A full list of regions can be found in the reference-data endpoint\n", "\n", "- 'format': choose which format to retrieve the data in, 'json' or 'csv'\n", "\n", "Both 'incoterm' and 'region' are optional endpoint parameters that are used to filter the data - if left undefined, the endpoint will just call all available posts on Hubs Cargo \n", "\n", "__N.B__ Metadata is only available via the JSON format." ] }, { "cell_type": "code", "execution_count": 4, "id": "dff2524b", "metadata": {}, "outputs": [], "source": [ "\n", "def fetch_active_posts(access_token, incoterm=None, region=None, format='json'):\n", "\n", " query_params = ''\n", "\n", " if incoterm is not None:\n", " query_params += '?incoterm={}'.format(incoterm)\n", " if region is not None:\n", " if len(query_params) >0:\n", " query_params += '®ion={}'.format(region)\n", " else:\n", " query_params += '?region={}'.format(region)\n", "\n", "\n", " uri=\"v1.0/lng/hubs/cargo/active/{}\".format(query_params)\n", " print(uri)\n", " \n", " content = do_api_get_query(\n", " uri=uri, access_token=access_token, format=format\n", " )\n", " \n", " if format == 'json':\n", " my_dict = content\n", " elif format == 'csv':\n", " # if there's no data to show, returns raw response (empty string) and \"No Data to Show\" message\n", " if len(content) == 0:\n", " my_dict = content\n", " print('No Data to Show')\n", " else:\n", " my_dict = content.decode('utf-8')\n", " my_dict = pd.read_csv(StringIO(my_dict)) # automatically converting into a Pandas DataFrame when choosing CSV format\n", " \n", " return my_dict" ] }, { "cell_type": "code", "execution_count": 15, "id": "6844b461", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "v1.0/lng/hubs/cargo/active/\n" ] }, { "data": { "text/plain": [ "{'terminals': {'00301fdd-c604-41e4-8d58-48148bde7aa9': 'Singapore',\n", " '00302863-23af-41a6-a8ec-82c703860811': 'Altamira',\n", " '0030505e-1e75-4f9c-b049-3ef0b9a1c089': 'Kochi',\n", " '003054f8-63cc-4344-9873-1de96c92da43': 'Wilhelmshaven 2',\n", " '00306490-345a-4528-a42c-d87c61e2c35b': 'Swinoujscie',\n", " '0030c461-9a63-403d-8f53-9327ea773517': 'Corpus Christi',\n", " '0030ca71-9252-44d5-9d34-3cee596ad1f6': 'Bontang',\n", " '0030d930-6574-4049-a739-327a16620429': 'Ravenna',\n", " '0030e65a-383d-4769-bc2b-33ae56657062': 'Brunsbuttel',\n", " '0030fe3d-20b4-4faf-9d8e-3e1c9b049f2e': 'Al Zour',\n", " '00312574-769c-4edf-9a55-362f3da20312': 'Gorgon',\n", " '00312f71-e530-4aa8-98a5-53e97f6afebd': 'Prelude',\n", " '00314d16-eada-4f37-bff3-d844708aeb45': 'Woodfibre LNG',\n", " '00316427-9e58-4ff1-8605-1042124ddcca': 'Gwangyang',\n", " '00316a05-9537-4fb4-993a-5e8b53929c49': 'Darwin',\n", " '00319332-b16d-479e-9098-0d19c578a931': 'Dahej',\n", " '0031b38c-2108-443e-9bea-bc0fccce37f7': 'Port Moresby',\n", " '0031c9b7-2130-4f2b-9e33-00ebc2c99d51': 'Aliaga',\n", " '0031ccda-8f17-4d48-9fb3-723cb1d23fb5': 'TVB (Spain Virtual Port)',\n", " '0031e113-6ef5-4a1f-bc25-64c2b40d4ea7': 'Pluto',\n", " '0031f139-2a77-45cc-ba99-a0deedc8a9ba': 'Cartagena',\n", " '00323df6-15b0-4e3f-93e1-afe3a3fe1b58': 'Rudong',\n", " '00324816-065e-48e9-94fa-98eaf1aee433': 'Eemshaven',\n", " '00325694-638e-4478-b01a-fc34f345b713': 'Incheon',\n", " '00325dcd-7649-4972-9746-77ecf3acd48c': 'QCLNG',\n", " '00326ebb-3243-49cb-ae12-21a528c54351': 'Sergipe',\n", " '0032fed7-dadd-4465-b8f5-0cc4e8915574': 'Shenzhen',\n", " '003304c5-8d7f-48fa-b3d5-b938b0ffb052': 'Pecem',\n", " '00332530-013f-42f2-a443-4ddaedbed242': 'Etki',\n", " '00333f74-dfa1-4d41-b161-80e038543d85': 'Port Kembla',\n", " '003342b7-ba5b-4f0e-b6df-4d95837a5691': 'Bintulu',\n", " '003349a0-0c2d-4517-bf0f-3aa7c47e3d50': 'Congo LNG',\n", " '003379a8-4358-4caf-99ef-0be7c5447700': 'Aqaba',\n", " '0033833f-3403-4397-97e3-2ccb9aedb62c': 'Argentina LNG',\n", " '003390dc-0918-485a-ba91-12465d2f1890': 'GLNG',\n", " '00339787-d27c-4605-8409-e1c544820cec': 'Atlantic LNG',\n", " '0033b569-7a8b-441b-841c-760e76d9e36e': 'Dunkerque',\n", " '0033d717-1d87-407b-94b0-3f115ecd1887': 'Kamchatka',\n", " '0033e22e-c545-469b-95c1-4d16bdbba1ef': 'Nusantara',\n", " '00348162-8284-447d-b641-1f06b9078fdd': 'Gate',\n", " '0034a830-e3b4-4613-8264-57225ef6abd8': 'Summit LNG',\n", " '0034aa21-2462-43bc-b96b-e4f985078bb7': 'Toscana',\n", " '0034d09b-a62d-4b1b-958c-c9e862594979': 'Ichthys',\n", " '0034fc62-feb4-44d6-b201-584a977ef3e9': 'Pengerang',\n", " '0034fc9c-bb57-42b6-a91b-6f08c3795a25': 'Bonny LNG',\n", " '00350607-0903-4828-8ea5-bdeb8649a31e': 'Barcelona',\n", " '003509bf-b256-455e-9480-22162a401587': 'Grain',\n", " '00351043-8269-4fba-919f-5d54d7d48915': 'Piombino',\n", " '00351464-a9ff-42b4-94a2-8e81e05154d3': 'Lubmin',\n", " '00352746-7b90-4f69-a995-6048f670c1b8': 'Alexandroupolis',\n", " '00352a22-e959-4233-b93d-d23a0da3dfed': 'Elba Island',\n", " '00353980-7951-434c-a45b-9ec5ef1735a0': 'Dragon',\n", " '003553b5-b442-4ec4-b1f7-d95cef95f4bd': 'Bahrain',\n", " '00356046-86ab-4a76-a689-99af1b3027ad': 'Cameron (Liqu.)',\n", " '00359de5-92cd-4abd-ac62-d63bef384c1b': 'Sines',\n", " '0035ad68-ff62-4899-8bd3-b8c2f04aae7a': 'Guanabara Bay',\n", " '0035c0fe-299d-4696-8c94-45f4878c9dda': 'Hazira',\n", " '00361e73-69ca-41be-a577-704f62adf3fa': 'Mundra',\n", " '003627f4-c752-4ce4-9ae7-7ad0fa6e53ce': 'Qalhat',\n", " '00364105-b12c-4f4d-94dc-1cd3069287b8': 'Zeebrugge',\n", " '0036961e-e634-4929-874d-45f53265ed7c': 'Rayong',\n", " '00369d3c-62db-4833-8c49-ea1a2fbb2b18': 'LNG Canada',\n", " '0036bb58-b8b6-4f04-95fe-ddce2a1f16f9': 'Caofeidian',\n", " '0036c796-2663-4062-b746-eecc658dbb5a': 'Dabhol',\n", " '0036cd2f-ff65-471e-8e6e-9c7cbae674b6': 'Rovigo',\n", " '0036dbaa-23dd-42cc-a22a-2ec332951bac': 'Bilbao',\n", " '0037496b-cd8a-45f1-875e-6ab93d0bef90': 'Saros',\n", " '00378915-9416-4fce-afae-8e55ea95b26d': 'Arun',\n", " '0037ae4c-39c4-4fd3-8e13-ac5e557add0a': 'Cameroon FLNG',\n", " '0037c692-6771-4cf2-8099-4e695e69935a': 'Ennore LNG',\n", " '0037ca5d-114e-4e66-a16e-f805562ee6dc': 'Damietta',\n", " '003801fb-0c33-4da6-9bec-8cefd706fd8c': 'Le Havre',\n", " '00381c87-4180-4430-80f1-bf828099124f': 'NWS',\n", " '00382738-2387-431b-8740-155d26a8ef0b': 'Quintero',\n", " '00383491-7e92-4b89-8b26-473fb3dcb9ed': 'Rio Grande LNG',\n", " '00386200-185c-449a-a86f-7eb68ce6bfc7': 'Yung An',\n", " '00386dd0-8fe0-4460-ac8c-4710d999a69e': 'Sagunto',\n", " '00387996-f338-4c38-abcf-3fcfd3712339': 'Tangguh',\n", " '0038c659-fc9c-4d7d-a136-b227aba287ba': 'DSLNG',\n", " '00390022-7056-45ec-a9af-c05de0145ec9': 'Covenas FSRU',\n", " '00390a6a-71bf-43e4-a4ba-8606e535d745': 'Gibraltar',\n", " '00391ef1-89ab-4645-a867-4a9e73f822fb': 'Mugardos',\n", " '00394ee5-30f9-48d3-abea-8e1858548ad1': 'GasPort LNG',\n", " '00398183-cb2e-4d28-b673-44289872ba94': 'Dapeng',\n", " '00398395-a9d6-4d09-8e17-f731caf760d9': 'Peru LNG',\n", " '00398967-3ee1-4b26-bcdb-805ad19dbcce': 'Wheatstone',\n", " '0039b921-cc5c-47e3-9a31-9dfb0c55d345': 'Freeport',\n", " '0039ba1c-540a-4198-94ac-0da6dc9cf25d': 'APLNG',\n", " '0039ea55-8adc-4448-ba3f-812c44c2d293': 'Bahia Blanca',\n", " '003a0415-de98-4fd2-86bb-0aaceee67b1f': 'Dhamra',\n", " '003a212d-0ecb-464a-a679-b90ea65a7a14': 'La Spezia',\n", " '003a2e76-b6e3-4fb3-9c3f-d12b547ec41e': 'Port Açu',\n", " '003a81ab-1590-4d5d-9a4c-69857a9f6985': 'Lampung',\n", " '003a8724-b074-42e2-9e2f-ac3cc3e67ea4': 'Yamal',\n", " '003a90f9-9cde-436f-9bec-4f9e786c2ea7': 'Tianjin',\n", " '003a9231-acd5-40a0-b1dd-8c7d65612eec': 'Inkoo',\n", " '003a9911-55ce-40e1-8606-e81dc19ae490': 'Montoir',\n", " '003aab65-41c1-4952-a9dd-d6114af8a3b2': 'Ain Sukhna 2',\n", " '003ad466-2aba-4f40-b016-f8af4762034f': 'Cartagena (COL)',\n", " '003adf4e-01b5-4680-9cd7-49613d5d0e3e': 'Plaquemines',\n", " '003af9a7-c419-4360-8fad-4a558a1d8730': 'Wilhelmshaven',\n", " '003afd77-2b63-41e4-8b7a-6d6294236d78': 'Bethioua',\n", " '003b319e-b29e-4853-b4ee-85794d5bacba': 'Stade',\n", " '003b48e3-aedd-417c-8ea3-2d1dd434d8a9': 'Das Island',\n", " '003ba54e-5841-4fb2-bae2-16d04b02b1ff': 'Dortyol FSRU',\n", " '003bdb09-9cc9-4998-a849-0e0dce812fa7': 'Soyo',\n", " '003bfc7e-1fad-4ec9-9fe9-d5b0889bef02': 'Klaipeda',\n", " '003c065f-2780-49f0-a070-b1313f782503': 'Deutsche Ostsee',\n", " '003c0df2-60d1-4aa6-89c1-c5a860c8d56f': 'Ereglisi',\n", " '003c12fa-0512-43f6-9ed0-04f1bb37fd4f': 'El Musel',\n", " '003c2da6-6a74-4e29-aef6-a789a747ac65': 'Futtsu',\n", " '003cc07a-14ff-4f9e-87e4-7124d809c045': 'Batangas',\n", " '003cf9e2-deeb-46cc-bd69-78103a444623': 'Krk LNG FSRU',\n", " '003d2908-3754-46a4-acdc-675e9f63ae06': 'South Hook',\n", " '003d3a18-7463-4aef-8c32-22054d6815af': 'Revithoussa',\n", " '003da60d-2e4f-450e-9a83-43e1ac3517fc': 'Port Qasim FSRU',\n", " '003dec0a-ce8f-41db-8c24-4d7ef6addf70': 'Sabine Pass',\n", " '003e005e-9aeb-4e31-b44f-6f1a5d79ea67': 'Calcasieu Pass',\n", " '003e049d-6197-4a0d-b075-517aaffd689a': 'Escobar',\n", " '003e20ed-3306-47c4-94bc-762cf3e896b9': 'Chittagong',\n", " '003e3e36-1826-42f8-b789-c8137d7e0f34': 'Fos Cavaou',\n", " '003e4ff1-6888-4d04-95e2-42e5b42ff722': 'Costa Azul',\n", " '003e50dc-2dca-44e5-b519-c4d704c6762d': 'Bioko',\n", " '003e5f6c-9157-44b0-87ce-b5c8600746d1': 'Dakar',\n", " '003e8539-3a98-48fa-b35d-0ba061beea4e': 'Cove Point',\n", " '003ea1c2-975a-4b03-a753-4a435e730288': 'Chhara',\n", " '003ea2f5-f0fa-4e5e-9717-734cefabec68': 'Richards Bay',\n", " '003ea9c4-f985-4fe1-b7a3-a833c142882b': 'Bahia',\n", " '003ef335-be59-48ca-97aa-cf1fd8dea6fb': 'Puerto Libertad',\n", " '003effde-1384-41fd-b604-15cc81167a3c': 'Coral South FLNG',\n", " '003f1759-2659-47b7-b5ff-72878a81e5eb': 'Zhoushan',\n", " '003f52cd-5b96-41b8-bc82-600f6f7499cc': 'Colon',\n", " '003f6682-9a02-442e-9762-f75b63530277': 'Huelva',\n", " '003f9252-e4e3-4957-8462-36f4b61b3370': 'Murmansk',\n", " '003f92ce-86d5-4d03-9761-311036c47812': 'Hammerfest',\n", " '003f9d1b-b4ad-4de9-8c8d-bd7fbcacd3df': 'Ras Laffan',\n", " '003fa7a4-b8ff-42e7-a146-983baddc769c': 'Delfin FLNG',\n", " '003fd3f3-1efd-4fcc-b0f2-4d6ab77c6902': 'Jebel Ali',\n", " '003ff22f-77d8-413f-9997-2c6280e7c28c': 'Lake Charles'}}" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# calling the data as a JSON\n", "data_json = fetch_active_posts(access_token, format='json')\n", "\n", "# printing metaData\n", "data_json['metaData']" ] }, { "cell_type": "code", "execution_count": 6, "id": "c8bc66b2", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "v1.0/lng/hubs/cargo/active/\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
PostIDIncotermHubRegionsInterestTypeOrderTypeStatusPosterOrgNameConnectionCountTerminalNamesTerminalsAreHidden...IndicativePriceIndexIndicativePriceIndexedMonthIndicativePriceIndexPercentageIndicativePriceIndexConstantIndicativePriceIsHiddenOtherInfoOtherInfoIsHiddenPostedAtUTCPostValidUntilUTCPostLastUpdatedAtUTC
0LNG-USG-S-0177fobusgswapofferedactiveAnonymous0Cameron (Liqu.)False...NaNNaNNaNNaNTruevol flexFalse2026-02-10T20:10:45.773575Z2026-02-14T00:00:00Z2026-02-10T20:10:45.773666Z
1LNG-USG-S-0177fobusgswaprequestedactiveAnonymous0Sabine Pass,Corpus Christi,Plaquemines,Calcasi...False...NaNNaNNaNNaNTruevol flexFalse2026-02-10T20:10:45.773575Z2026-02-14T00:00:00Z2026-02-10T20:10:45.773666Z
2LNG-USG-S-0178fobusgswapofferedactiveAnonymous0Sabine PassFalse...NaNNaNNaNNaNFalsevol flexFalse2026-02-10T20:12:11.088240Z2026-02-14T00:00:00Z2026-02-10T20:12:11.088325Z
\n", "

3 rows × 34 columns

\n", "
" ], "text/plain": [ " PostID Incoterm HubRegions InterestType OrderType Status \\\n", "0 LNG-USG-S-0177 fob usg swap offered active \n", "1 LNG-USG-S-0177 fob usg swap requested active \n", "2 LNG-USG-S-0178 fob usg swap offered active \n", "\n", " PosterOrgName ConnectionCount \\\n", "0 Anonymous 0 \n", "1 Anonymous 0 \n", "2 Anonymous 0 \n", "\n", " TerminalNames TerminalsAreHidden ... \\\n", "0 Cameron (Liqu.) False ... \n", "1 Sabine Pass,Corpus Christi,Plaquemines,Calcasi... False ... \n", "2 Sabine Pass False ... \n", "\n", " IndicativePriceIndex IndicativePriceIndexedMonth \\\n", "0 NaN NaN \n", "1 NaN NaN \n", "2 NaN NaN \n", "\n", " IndicativePriceIndexPercentage IndicativePriceIndexConstant \\\n", "0 NaN NaN \n", "1 NaN NaN \n", "2 NaN NaN \n", "\n", " IndicativePriceIsHidden OtherInfo OtherInfoIsHidden \\\n", "0 True vol flex False \n", "1 True vol flex False \n", "2 False vol flex False \n", "\n", " PostedAtUTC PostValidUntilUTC \\\n", "0 2026-02-10T20:10:45.773575Z 2026-02-14T00:00:00Z \n", "1 2026-02-10T20:10:45.773575Z 2026-02-14T00:00:00Z \n", "2 2026-02-10T20:12:11.088240Z 2026-02-14T00:00:00Z \n", "\n", " PostLastUpdatedAtUTC \n", "0 2026-02-10T20:10:45.773666Z \n", "1 2026-02-10T20:10:45.773666Z \n", "2 2026-02-10T20:12:11.088325Z \n", "\n", "[3 rows x 34 columns]" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# calling data as a CSV\n", "live_df = fetch_active_posts(access_token, format='csv')\n", "live_df.head(3)" ] }, { "cell_type": "code", "execution_count": 7, "id": "a1df605f", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Index(['PostID', 'Incoterm', 'HubRegions', 'InterestType', 'OrderType',\n", " 'Status', 'PosterOrgName', 'ConnectionCount', 'TerminalNames',\n", " 'TerminalsAreHidden', 'WindowStartDate', 'WindowEndDate',\n", " 'WindowOutrightCargoCount', 'WindowsAreHidden', 'WindowsAreFlexible',\n", " 'QuantityValue', 'QuantityUnit', 'QuantityTolerancePercent',\n", " 'QuantityIsHidden', 'QuantityIsFlexible',\n", " 'SwapPaymentIndicationPreference',\n", " 'SwapPaymentIndicationPreferenceIsHidden', 'IndicativePriceStructure',\n", " 'IndicativePriceFixedValue', 'IndicativePriceIndex',\n", " 'IndicativePriceIndexedMonth', 'IndicativePriceIndexPercentage',\n", " 'IndicativePriceIndexConstant', 'IndicativePriceIsHidden', 'OtherInfo',\n", " 'OtherInfoIsHidden', 'PostedAtUTC', 'PostValidUntilUTC',\n", " 'PostLastUpdatedAtUTC'],\n", " dtype='object')" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "live_df.columns" ] }, { "cell_type": "markdown", "id": "e3bb7625", "metadata": {}, "source": [ "# 4. Inactive (i.e. Historical) Slots\n", "\n", "Here we define the function used to call the inactive Hubs Cargo posts. This function takes three parameters:\n", "\n", "- 'incoterm': define which incoterm you'd like to filter by, 'fob' or 'des'\n", "\n", "- 'region': choose which region you'd like to filter by. A full list of regions can be found in the reference-data endpoint\n", "\n", "- 'format': choose which format to retrieve the data in, 'json' or 'csv'\n", "\n", "Both 'incoterm' and 'region' are optional endpoint parameters that are used to filter the data - if left undefined, the endpoint will just call all available posts on Hubs Cargo \n", "\n", "__N.B__ Metadata is only available via the JSON format." ] }, { "cell_type": "code", "execution_count": 8, "id": "ea4e7fc3", "metadata": {}, "outputs": [], "source": [ "## Defining the function\n", "\n", "def fetch_inactive_posts(access_token, incoterm=None, region=None, format='json'):\n", "\n", " query_params = ''\n", "\n", " if incoterm is not None:\n", " query_params += '?incoterm={}'.format(incoterm)\n", " if region is not None:\n", " if len(query_params) >0:\n", " query_params += '®ion={}'.format(region)\n", " else:\n", " query_params += '?region={}'.format(region)\n", "\n", "\n", " uri=\"v1.0/lng/hubs/cargo/inactive/{}\".format(query_params)\n", " print(uri)\n", " \n", " content = do_api_get_query(\n", " uri=uri, access_token=access_token, format=format\n", " )\n", " \n", " if format == 'json':\n", " my_dict = content\n", " elif format == 'csv':\n", " # if there's no data to show, returns raw response (empty string) and \"No Data to Show\" message\n", " if len(content) == 0:\n", " my_dict = content\n", " print('No Data to Show')\n", " else:\n", " my_dict = content.decode('utf-8')\n", " my_dict = pd.read_csv(StringIO(my_dict)) # automatically converting into a Pandas DataFrame when choosing CSV format\n", " \n", " return my_dict" ] }, { "cell_type": "code", "execution_count": 9, "id": "38327f98", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "v1.0/lng/hubs/cargo/inactive/\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
PostIDIncotermHubRegionsInterestTypeOrderTypeStatusPosterOrgNameConnectionCountTerminalNamesTerminalsAreHidden...IndicativePriceIndexIndicativePriceIndexedMonthIndicativePriceIndexPercentageIndicativePriceIndexConstantIndicativePriceIsHiddenOtherInfoOtherInfoIsHiddenPostedAtUTCPostValidUntilUTCPostLastUpdatedAtUTC
0LNG-USG-S-0175fobusgswapofferedexpiredAnonymous0Calcasieu PassFalse...NaNNaNNaNNaNTrueNaNFalse2026-02-04T14:07:04.756076Z2026-02-11T00:00:00Z2026-02-11T00:00:06.614503Z
1LNG-USG-S-0175fobusgswaprequestedexpiredAnonymous0Sabine Pass,Corpus Christi,Plaquemines,Calcasi...False...NaNNaNNaNNaNTruevol flexFalse2026-02-04T14:07:04.756076Z2026-02-11T00:00:00Z2026-02-11T00:00:06.614503Z
2LNG-NWE-O-0042desnwe,sweoutrightofferexpiredAnonymous0Gate,Zeebrugge,Dunkerque,Bilbao,Sines,Sagunto,...False...TTF0.0NaNNaNFalseNaNFalse2026-01-30T14:52:48.090799Z2026-02-03T18:00:00Z2026-02-03T18:00:05.529188Z
3LNG-USG-O-0056fobusgoutrightofferexpiredAnonymous0Sabine Pass,Corpus Christi,Plaquemines,Calcasi...False...TTF0.0NaNNaNFalseNaNFalse2026-01-29T16:19:57.695064Z2026-02-06T17:00:00Z2026-02-06T17:00:05.762594Z
4LNG-USG-S-0170fobusgswapofferedexpiredAnonymous0Calcasieu PassFalse...NaNNaNNaNNaNFalseNaNFalse2026-01-29T14:59:16.285599Z2026-01-30T17:00:00Z2026-01-30T17:00:03.101753Z
\n", "

5 rows × 34 columns

\n", "
" ], "text/plain": [ " PostID Incoterm HubRegions InterestType OrderType Status \\\n", "0 LNG-USG-S-0175 fob usg swap offered expired \n", "1 LNG-USG-S-0175 fob usg swap requested expired \n", "2 LNG-NWE-O-0042 des nwe,swe outright offer expired \n", "3 LNG-USG-O-0056 fob usg outright offer expired \n", "4 LNG-USG-S-0170 fob usg swap offered expired \n", "\n", " PosterOrgName ConnectionCount \\\n", "0 Anonymous 0 \n", "1 Anonymous 0 \n", "2 Anonymous 0 \n", "3 Anonymous 0 \n", "4 Anonymous 0 \n", "\n", " TerminalNames TerminalsAreHidden ... \\\n", "0 Calcasieu Pass False ... \n", "1 Sabine Pass,Corpus Christi,Plaquemines,Calcasi... False ... \n", "2 Gate,Zeebrugge,Dunkerque,Bilbao,Sines,Sagunto,... False ... \n", "3 Sabine Pass,Corpus Christi,Plaquemines,Calcasi... False ... \n", "4 Calcasieu Pass False ... \n", "\n", " IndicativePriceIndex IndicativePriceIndexedMonth \\\n", "0 NaN NaN \n", "1 NaN NaN \n", "2 TTF 0.0 \n", "3 TTF 0.0 \n", "4 NaN NaN \n", "\n", " IndicativePriceIndexPercentage IndicativePriceIndexConstant \\\n", "0 NaN NaN \n", "1 NaN NaN \n", "2 NaN NaN \n", "3 NaN NaN \n", "4 NaN NaN \n", "\n", " IndicativePriceIsHidden OtherInfo OtherInfoIsHidden \\\n", "0 True NaN False \n", "1 True vol flex False \n", "2 False NaN False \n", "3 False NaN False \n", "4 False NaN False \n", "\n", " PostedAtUTC PostValidUntilUTC \\\n", "0 2026-02-04T14:07:04.756076Z 2026-02-11T00:00:00Z \n", "1 2026-02-04T14:07:04.756076Z 2026-02-11T00:00:00Z \n", "2 2026-01-30T14:52:48.090799Z 2026-02-03T18:00:00Z \n", "3 2026-01-29T16:19:57.695064Z 2026-02-06T17:00:00Z \n", "4 2026-01-29T14:59:16.285599Z 2026-01-30T17:00:00Z \n", "\n", " PostLastUpdatedAtUTC \n", "0 2026-02-11T00:00:06.614503Z \n", "1 2026-02-11T00:00:06.614503Z \n", "2 2026-02-03T18:00:05.529188Z \n", "3 2026-02-06T17:00:05.762594Z \n", "4 2026-01-30T17:00:03.101753Z \n", "\n", "[5 rows x 34 columns]" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# calling the function defined above\n", "hist_df = fetch_inactive_posts(access_token, format='csv')\n", "hist_df.head(5)" ] }, { "cell_type": "code", "execution_count": 12, "id": "2035d298", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "v1.0/lng/hubs/cargo/inactive/?incoterm=fob®ion=usg\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
PostIDIncotermHubRegionsInterestTypeOrderTypeStatusPosterOrgNameConnectionCountTerminalNamesTerminalsAreHidden...IndicativePriceIndexIndicativePriceIndexedMonthIndicativePriceIndexPercentageIndicativePriceIndexConstantIndicativePriceIsHiddenOtherInfoOtherInfoIsHiddenPostedAtUTCPostValidUntilUTCPostLastUpdatedAtUTC
0LNG-USG-S-0175fobusgswaprequestedexpiredAnonymous0Sabine Pass,Corpus Christi,Plaquemines,Calcasi...False...NaNNaNNaNNaNTruevol flexFalse2026-02-04T14:07:04.756076Z2026-02-11T00:00:00Z2026-02-11T00:00:06.614503Z
1LNG-USG-O-0056fobusgoutrightofferexpiredAnonymous0Sabine Pass,Corpus Christi,Plaquemines,Calcasi...False...TTF0.0NaNNaNFalseNaNFalse2026-01-29T16:19:57.695064Z2026-02-06T17:00:00Z2026-02-06T17:00:05.762594Z
2LNG-USG-S-0170fobusg,wafswaprequestedexpiredAnonymous0Sabine Pass,Corpus Christi,Calcasieu Pass,Free...False...NaNNaNNaNNaNFalseNaNFalse2026-01-29T14:59:16.285599Z2026-01-30T17:00:00Z2026-01-30T17:00:03.101753Z
3LNG-USG-S-0169fobusgswaprequestedexpiredAnonymous0Sabine Pass,Corpus Christi,Plaquemines,Calcasi...False...NaNNaNNaNNaNFalseNaNFalse2026-01-27T18:05:33.520171Z2026-02-02T17:00:00Z2026-02-02T17:00:02.227684Z
4LNG-USG-S-0167fobusgswaprequestedexpiredAnonymous0Sabine Pass,Corpus Christi,Plaquemines,Calcasi...False...NaNNaNNaNNaNFalseNaNFalse2026-01-26T15:04:24.117462Z2026-01-30T18:00:00Z2026-01-30T18:00:01.908710Z
\n", "

5 rows × 34 columns

\n", "
" ], "text/plain": [ " PostID Incoterm HubRegions InterestType OrderType Status \\\n", "0 LNG-USG-S-0175 fob usg swap requested expired \n", "1 LNG-USG-O-0056 fob usg outright offer expired \n", "2 LNG-USG-S-0170 fob usg,waf swap requested expired \n", "3 LNG-USG-S-0169 fob usg swap requested expired \n", "4 LNG-USG-S-0167 fob usg swap requested expired \n", "\n", " PosterOrgName ConnectionCount \\\n", "0 Anonymous 0 \n", "1 Anonymous 0 \n", "2 Anonymous 0 \n", "3 Anonymous 0 \n", "4 Anonymous 0 \n", "\n", " TerminalNames TerminalsAreHidden ... \\\n", "0 Sabine Pass,Corpus Christi,Plaquemines,Calcasi... False ... \n", "1 Sabine Pass,Corpus Christi,Plaquemines,Calcasi... False ... \n", "2 Sabine Pass,Corpus Christi,Calcasieu Pass,Free... False ... \n", "3 Sabine Pass,Corpus Christi,Plaquemines,Calcasi... False ... \n", "4 Sabine Pass,Corpus Christi,Plaquemines,Calcasi... False ... \n", "\n", " IndicativePriceIndex IndicativePriceIndexedMonth \\\n", "0 NaN NaN \n", "1 TTF 0.0 \n", "2 NaN NaN \n", "3 NaN NaN \n", "4 NaN NaN \n", "\n", " IndicativePriceIndexPercentage IndicativePriceIndexConstant \\\n", "0 NaN NaN \n", "1 NaN NaN \n", "2 NaN NaN \n", "3 NaN NaN \n", "4 NaN NaN \n", "\n", " IndicativePriceIsHidden OtherInfo OtherInfoIsHidden \\\n", "0 True vol flex False \n", "1 False NaN False \n", "2 False NaN False \n", "3 False NaN False \n", "4 False NaN False \n", "\n", " PostedAtUTC PostValidUntilUTC \\\n", "0 2026-02-04T14:07:04.756076Z 2026-02-11T00:00:00Z \n", "1 2026-01-29T16:19:57.695064Z 2026-02-06T17:00:00Z \n", "2 2026-01-29T14:59:16.285599Z 2026-01-30T17:00:00Z \n", "3 2026-01-27T18:05:33.520171Z 2026-02-02T17:00:00Z \n", "4 2026-01-26T15:04:24.117462Z 2026-01-30T18:00:00Z \n", "\n", " PostLastUpdatedAtUTC \n", "0 2026-02-11T00:00:06.614503Z \n", "1 2026-02-06T17:00:05.762594Z \n", "2 2026-01-30T17:00:03.101753Z \n", "3 2026-02-02T17:00:02.227684Z \n", "4 2026-01-30T18:00:01.908710Z \n", "\n", "[5 rows x 34 columns]" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# calling the function using the optional \"incoterm\" and \"region\" parameters\n", "usgc_df = fetch_inactive_posts(access_token, incoterm='fob', region='usg', format='csv')\n", "usgc_df.head(5)" ] } ], "metadata": { "kernelspec": { "display_name": "base", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.5" } }, "nbformat": 4, "nbformat_minor": 5 }