{ "cells": [ { "cell_type": "markdown", "id": "92475c69", "metadata": {}, "source": [ "# Python API Example - JKM-TTF vs Arb Breakevens\n", "\n", "Here we plot JKM-TTF prices against relevant Arb Breakeven levels. This script can also be used for TTF prices vs TTF breakevens.\n", "\n", "### Have any questions?\n", "\n", "If you have any questions regarding our API, or need help accessing specific datasets, please contact us at:\n", "\n", "__data@sparkcommodities.com__\n", "\n", "or refer to our API website for more information about this endpoint:\n", "https://www.sparkcommodities.com/api/" ] }, { "cell_type": "markdown", "id": "c5716130", "metadata": {}, "source": [ "## 1. Importing Data\n", "\n", "Here we define the functions that allow us to retrieve the valid credentials to access the Spark API.\n", "\n", "This section can remain unchanged for most Spark API users." ] }, { "cell_type": "code", "execution_count": null, "id": "33fb0640", "metadata": {}, "outputs": [], "source": [ "import json\n", "import os\n", "import sys\n", "import pandas as pd\n", "import numpy as np\n", "from base64 import b64encode\n", "from pprint import pprint\n", "from urllib.parse import urljoin\n", "import datetime\n", "from io import StringIO\n", "\n", "\n", "try:\n", " from urllib import request, parse\n", " from urllib.error import HTTPError\n", "except ImportError:\n", " raise RuntimeError(\"Python 3 required\")\n", "\n", "\n", "API_BASE_URL = \"https://api.sparkcommodities.com\"\n", "\n", "\n", "def retrieve_credentials(file_path=None):\n", " \"\"\"\n", " Find credentials either by reading the client_credentials file or reading\n", " environment variables\n", " \"\"\"\n", " if file_path is None:\n", " client_id = os.getenv(\"SPARK_CLIENT_ID\")\n", " client_secret = os.getenv(\"SPARK_CLIENT_SECRET\")\n", " if not client_id or not client_secret:\n", " raise RuntimeError(\n", " \"SPARK_CLIENT_ID and SPARK_CLIENT_SECRET environment vars required\"\n", " )\n", " else:\n", " # Parse the file\n", " if not os.path.isfile(file_path):\n", " raise RuntimeError(\"The file {} doesn't exist\".format(file_path))\n", "\n", " with open(file_path) as fp:\n", " lines = [l.replace(\"\\n\", \"\") for l in fp.readlines()]\n", "\n", " if lines[0] in (\"clientId,clientSecret\", \"client_id,client_secret\"):\n", " client_id, client_secret = lines[1].split(\",\")\n", " else:\n", " print(\"First line read: '{}'\".format(lines[0]))\n", " raise RuntimeError(\n", " \"The specified file {} doesn't look like to be a Spark API client \"\n", " \"credentials file\".format(file_path)\n", " )\n", "\n", " print(\">>>> Found credentials!\")\n", " print(\n", " \">>>> Client_id={}, client_secret={}****\".format(client_id, client_secret[:5])\n", " )\n", "\n", " return client_id, client_secret\n", "\n", "\n", "def do_api_post_query(uri, body, headers):\n", " url = urljoin(API_BASE_URL, uri)\n", "\n", " data = json.dumps(body).encode(\"utf-8\")\n", "\n", " # HTTP POST request\n", " req = request.Request(url, data=data, headers=headers)\n", " try:\n", " response = request.urlopen(req)\n", " except HTTPError as e:\n", " print(\"HTTP Error: \", e.code)\n", " print(e.read())\n", " sys.exit(1)\n", "\n", " resp_content = response.read()\n", "\n", " # The server must return HTTP 201. Raise an error if this is not the case\n", " assert response.status == 201, resp_content\n", "\n", " # The server returned a JSON response\n", " content = json.loads(resp_content)\n", "\n", " return content\n", "\n", "\n", "def do_api_get_query(uri, access_token, format='json'):\n", " \"\"\"\n", " After receiving an Access Token, we can request information from the API.\n", " \"\"\"\n", " url = urljoin(API_BASE_URL, uri)\n", "\n", " if format == 'json':\n", " headers = {\n", " \"Authorization\": \"Bearer {}\".format(access_token),\n", " \"Accept\": \"application/json\",\n", " }\n", " elif format == 'csv':\n", " headers = {\n", " \"Authorization\": \"Bearer {}\".format(access_token),\n", " \"Accept\": \"text/csv\"\n", " }\n", " else:\n", " raise ValueError('The format parameter only takes `csv` or `json` as inputs')\n", "\n", " # HTTP GET request\n", " req = request.Request(url, headers=headers)\n", " try:\n", " response = request.urlopen(req)\n", " except HTTPError as e:\n", " print(\"HTTP Error: \", e.code)\n", " print(e.read())\n", " sys.exit(1)\n", "\n", " resp_content = response.read()\n", " #status = response.status\n", "\n", " # The server must return HTTP 200. Raise an error if this is not the case\n", " assert response.status == 200, resp_content\n", "\n", " # Storing response based on requested format\n", " if format == 'json':\n", " content = json.loads(resp_content)\n", " elif format == 'csv':\n", " content = resp_content\n", "\n", " return content\n", "\n", "\n", "def get_access_token(client_id, client_secret):\n", " \"\"\"\n", " Get a new access_token. Access tokens are the thing that applications use to make\n", " API requests. Access tokens must be kept confidential in storage.\n", "\n", " # Procedure:\n", "\n", " Do a POST query with `grantType` in the body. A basic authorization\n", " HTTP header is required. The \"Basic\" HTTP authentication scheme is defined in\n", " RFC 7617, which transmits credentials as `clientId:clientSecret` pairs, encoded\n", " using base64.\n", " \"\"\"\n", "\n", " # Note: for the sake of this example, we choose to use the Python urllib from the\n", " # standard lib. One should consider using https://requests.readthedocs.io/\n", "\n", " payload = \"{}:{}\".format(client_id, client_secret).encode()\n", " headers = {\n", " \"Authorization\": \"Basic {}\".format(b64encode(payload).decode()),\n", " \"Accept\": \"application/json\",\n", " \"Content-Type\": \"application/json\",\n", " }\n", " body = {\n", " \"grantType\": \"clientCredentials\",\n", " }\n", "\n", " content = do_api_post_query(uri=\"/oauth/token/\", body=body, headers=headers)\n", "\n", " print(\n", " \">>>> Successfully fetched an access token {}****, valid {} seconds.\".format(\n", " content[\"accessToken\"][:5], content[\"expiresIn\"]\n", " )\n", " )\n", "\n", " return content[\"accessToken\"]\n" ] }, { "cell_type": "markdown", "id": "fd3171a8", "metadata": {}, "source": [ "## N.B. Credentials\n", "\n", "Here we call the above functions, and input the file path to our credentials.\n", "\n", "N.B. You must have downloaded your client credentials CSV file before proceeding. Please refer to the API documentation if you have not dowloaded them already.\n", "\n", "The code then prints the available prices that are callable from the API, and their corresponding Python ticker names are displayed as a list at the bottom of the Output." ] }, { "cell_type": "code", "execution_count": null, "id": "fd7e89bf", "metadata": {}, "outputs": [], "source": [ "# Insert file path to your client credentials here\n", "client_id, client_secret = retrieve_credentials(file_path=\"/tmp/client_credentials.csv\")\n", "\n", "# Authenticate:\n", "access_token = get_access_token(client_id, client_secret)" ] }, { "cell_type": "markdown", "id": "10bafcf1", "metadata": {}, "source": [ "# 2. Arb Breakevens Reference Data\n", "\n", "The Reference Data endpoint returns the UUID codes associated with each FoB Port available on the Arb Breakevens endpoint. \n", "\n", "If there are two arb breakevens available for a single FoB Port, e.g. \"Sabine Pass via COGH\" or \"Sabine Pass via Panama\", these are included as separate entries in the Reference Data response. The FoB Port UUID remains the same between these two entries. " ] }, { "cell_type": "code", "execution_count": null, "id": "4d211cdb", "metadata": {}, "outputs": [], "source": [ "# define reference data function\n", "\n", "def fetch_reference_data(access_token, format='json'):\n", "\n", " uri=\"/v1.0/intraday/breakevens/reference-data/\"\n", " print(uri)\n", " \n", " content = do_api_get_query(\n", " uri, access_token=access_token, format=format\n", " )\n", "\n", " if format == 'json':\n", " data = content\n", " elif format == 'csv':\n", " # if there's no data to show, returns raw response (empty string) and \"No Data to Show\" message\n", " if len(content) == 0:\n", " data = pd.DataFrame()\n", " print('No Data to Show')\n", " else:\n", " data = content.decode('utf-8')\n", " data = pd.read_csv(StringIO(data)) # automatically converting into a Pandas DataFrame when choosing CSV format\n", "\n", " return data\n", "\n", "\n", "# call data\n", "ref_df = fetch_reference_data(access_token, format='csv')\n", "ref_df" ] }, { "cell_type": "markdown", "id": "e27e05fb", "metadata": {}, "source": [ "# 3. Endpoint Fetch Functions\n", "\n", "## 3.1 Arb Breakevens\n", "\n", "This endpoint takes 7 parameters - details on these parameters can be found on the API docs website:\n", "\n", "https://www.sparkcommodities.com/api/intraday/breakevens.html" ] }, { "cell_type": "code", "execution_count": null, "id": "24adff55", "metadata": {}, "outputs": [], "source": [ "## Defining the function\n", "\n", "\n", "def fetch_historical_breakevens(access_token, breakeven_type, ticker, via, percent_hire, unit, start, end, format='json'):\n", " \n", " query_params = \"?breakeven-type={}\".format(breakeven_type)\n", " query_params += \"&fob-port={}\".format(ticker)\n", " query_params += \"&via-point={}\".format(via)\n", " query_params += \"&percent-hire={}\".format(percent_hire)\n", " query_params += \"&unit={}\".format(unit)\n", " \n", "\n", " query_params += \"&start={}\".format(start)\n", " query_params += \"&end={}\".format(end)\n", "\n", " uri=\"/v1.0/intraday/breakevens/historical/{}\".format(query_params)\n", " print(uri)\n", " \n", " content = do_api_get_query(\n", " uri=uri, access_token=access_token, format=format\n", " )\n", "\n", " if format == 'json':\n", " data = content\n", " elif format == 'csv':\n", " # if there's no data to show, returns raw response (empty string) and \"No Data to Show\" message\n", " if len(content) == 0:\n", " data = pd.DataFrame()\n", " print('No Data to Show')\n", " else:\n", " data = content.decode('utf-8')\n", " data = pd.read_csv(StringIO(data)) # automatically converting into a Pandas DataFrame when choosing CSV format\n", " data['AsOf'] = pd.to_datetime(data['AsOf'], utc=True).dt.tz_localize(None)\n", "\n", " return data\n" ] }, { "cell_type": "markdown", "id": "8cc9fec8", "metadata": {}, "source": [ "## 3.2 Contracts Data (JKM-TTF or TTF)\n", "\n", "This endpoint takes 4 required parameters - details on these parameters can be found on our API docs website:\n", "\n", "https://www.sparkcommodities.com/api/intraday/contracts.html" ] }, { "cell_type": "code", "execution_count": null, "id": "97554539", "metadata": {}, "outputs": [], "source": [ "## Defining the function\n", "\n", "def fetch_historical_contracts(access_token, contract, unit, start, end, format='json'):\n", " \n", " query_params = \"?contract={}\".format(contract)\n", " query_params += \"&unit={}\".format(unit)\n", "\n", " query_params += \"&start={}\".format(start)\n", " query_params += \"&end={}\".format(end)\n", "\n", " uri=\"/v1.0/intraday/contracts/historical/{}\".format(query_params)\n", " print(uri)\n", " \n", " content = do_api_get_query(\n", " uri=uri, access_token=access_token, format=format\n", " )\n", "\n", " if format == 'json':\n", " data = content\n", " elif format == 'csv':\n", " # if there's no data to show, returns raw response (empty string) and \"No Data to Show\" message\n", " if len(content) == 0:\n", " data = pd.DataFrame()\n", " print('No Data to Show')\n", " else:\n", " data = content.decode('utf-8')\n", " data = pd.read_csv(StringIO(data)) # automatically converting into a Pandas DataFrame when choosing CSV format\n", "\n", " return data\n" ] }, { "cell_type": "markdown", "id": "09f6ccc4", "metadata": {}, "source": [ "# 4. Historical Data Function\n", "\n", "Currently, a maximum of 20 historical days can be called at one time due to the size of the data file. If more data points are required, the below code can be used, which splits the requested date range into looped calls of 20 historical days at a time.\n", "\n", "This function is applicable to both the Contracts and Breakevens endpoints - when calling the function, just specify which endpoint ('contracts' or 'breakevens') it's being used for. \n", "\n", "The 'start', 'end' and 'unit' parameters are applicable to both endpoints. All other parameters are contract specific - please refer to the documentation or the api_fetch functions above to see which parameters are applicable to each endpoint" ] }, { "cell_type": "code", "execution_count": null, "id": "aa3dd528", "metadata": {}, "outputs": [], "source": [ "def loop_historical_data(access_token, endpoint, contract=None, breakeven_type=None, ticker=None, via=None, percent_hire=None, unit=None, hist_start=None, hist_end=None):\n", " \n", " # error handling\n", " if endpoint not in {\"breakevens\", \"contracts\"}:\n", " raise ValueError(\"endpoint must be either 'breakevens' or 'contracts'\")\n", " if not hist_start or not hist_end:\n", " raise ValueError(\"hist_start and hist_end are required (YYYY-MM-DD)\")\n", "\n", " start_dt = datetime.datetime.strptime(hist_start, '%Y-%m-%d')\n", " end_dt = datetime.datetime.strptime(hist_end, '%Y-%m-%d')\n", " if end_dt < start_dt:\n", " raise ValueError(\"hist_end must be on or after hist_start\")\n", "\n", " # calculating the amount of days to retrieve data for\n", " hist_diff = (end_dt - start_dt).days\n", " \n", " t = 0\n", "\n", " starts = []\n", " ends = []\n", " \n", " # iterating through the historical data, 20 days at a time\n", " while t < hist_diff:\n", " # first loop & initialising the historical Dataframe\n", " if t == 0 and hist_diff >= 21:\n", " diff_end = datetime.datetime.strftime(datetime.datetime.strptime(hist_start, '%Y-%m-%d') + pd.Timedelta(20, unit='days'), '%Y-%m-%d')\n", " if endpoint == 'breakevens':\n", " hist_df = fetch_historical_breakevens(access_token, breakeven_type=breakeven_type, ticker=ticker, via=via, \n", " percent_hire=percent_hire, unit=unit, start=hist_start, end=diff_end, format='csv')\n", " elif endpoint == 'contracts':\n", " hist_df = fetch_historical_contracts(access_token, contract=contract, unit=unit, start=hist_start, end=diff_end, format='csv')\n", "\n", " starts.append(hist_start)\n", " ends.append(diff_end)\n", " \n", " # in case this function is used for a date range of < 20 days\n", " elif t==0 and hist_diff < 21:\n", " if endpoint == 'breakevens':\n", " hist_df = fetch_historical_breakevens(access_token, breakeven_type=breakeven_type, ticker=ticker, via=via, \n", " percent_hire=percent_hire, unit=unit, start=hist_start, end=hist_end, format='csv')\n", " elif endpoint == 'contracts':\n", " hist_df = fetch_historical_contracts(access_token, contract=contract, unit=unit, start=hist_start, end=hist_end, format='csv')\n", "\n", " # all other loops\n", " else:\n", " # calling data for the next 20 days\n", " if t < hist_diff-20:\n", " diff_start = datetime.datetime.strftime(datetime.datetime.strptime(hist_start, '%Y-%m-%d') + pd.Timedelta(t, unit='days'), '%Y-%m-%d')\n", " diff_end = datetime.datetime.strftime(datetime.datetime.strptime(diff_start, '%Y-%m-%d') + pd.Timedelta(20, unit='days'), '%Y-%m-%d')\n", " if endpoint == 'breakevens':\n", " hist2 = fetch_historical_breakevens(access_token, breakeven_type=breakeven_type, ticker=ticker, via=via, \n", " percent_hire=percent_hire, unit=unit, start=diff_start, end=diff_end, format='csv')\n", " elif endpoint == 'contracts':\n", " hist2 = fetch_historical_contracts(access_token, contract=contract, unit=unit, start=diff_start, end=diff_end, format='csv')\n", " hist_df = pd.concat([hist_df,hist2])\n", " starts.append(hist_start)\n", " ends.append(diff_end)\n", " \n", " # if <20 days until the end of the date range\n", " else:\n", " diff_start = datetime.datetime.strftime(datetime.datetime.strptime(hist_start, '%Y-%m-%d') + pd.Timedelta(t, unit='days'), '%Y-%m-%d')\n", " diff_end = datetime.datetime.strftime(datetime.datetime.strptime(diff_start, '%Y-%m-%d') + pd.Timedelta(hist_diff-t, unit='days'), '%Y-%m-%d')\n", " if endpoint == 'breakevens':\n", " hist2 = fetch_historical_breakevens(access_token, breakeven_type=breakeven_type, ticker=ticker, via=via, \n", " percent_hire=percent_hire, unit=unit, start=diff_start, end=diff_end, format='csv')\n", " elif endpoint == 'contracts':\n", " hist2 = fetch_historical_contracts(access_token, contract=contract, unit=unit, start=diff_start, end=diff_end, format='csv')\n", " hist_df = pd.concat([hist_df,hist2])\n", " starts.append(hist_start)\n", " ends.append(diff_end)\n", " \n", " t += 20\n", "\n", " hist_df['Value'] = pd.to_numeric(hist_df['Value'])\n", " hist_df['AsOf'] = pd.to_datetime(hist_df['AsOf'], utc=True).dt.tz_localize(None)\n", "\n", " return hist_df" ] }, { "cell_type": "markdown", "id": "c6f3a7d5", "metadata": {}, "source": [ "# 5. Calling the Historical Data" ] }, { "cell_type": "code", "execution_count": null, "id": "5f24a521", "metadata": {}, "outputs": [], "source": [ "# Save your chosen FoB Port UUIDs as a variable\n", "sab_uuid = ref_df[ref_df['FobPortName'] == 'Sabine Pass']['FobPortUuid'].iloc[0]\n", "bon_uuid = ref_df[ref_df['FobPortName'] == 'Bonny LNG']['FobPortUuid'].iloc[0]" ] }, { "cell_type": "code", "execution_count": null, "id": "d6010f4f", "metadata": {}, "outputs": [], "source": [ "# Calling the extended historical data function\n", "\n", "start = '2025-10-01'\n", "today = datetime.date.today().strftime('%Y-%m-%d')\n", "\n", "# Sabine Pass Arb Breakeven\n", "sab_df = loop_historical_data(access_token, endpoint='breakevens', breakeven_type='jkm-ttf', ticker=sab_uuid, via='cogh', \n", " percent_hire=100, unit='usd-per-mmbtu',\n", " hist_start=start, hist_end=today)\n", "\n", "# Sabine Pass Arb (Freight Considered Sunk) Breakeven\n", "sab0_df = loop_historical_data(access_token, endpoint='breakevens', breakeven_type='jkm-ttf', ticker=sab_uuid, via='cogh', \n", " percent_hire=0, unit='usd-per-mmbtu',\n", " hist_start=start, hist_end=today)\n", "\n", "# Bonny LNG Arb Breakeven\n", "bon_df = loop_historical_data(access_token, endpoint='breakevens', breakeven_type='jkm-ttf', ticker=bon_uuid, via='cogh', \n", " percent_hire=100, unit='usd-per-mmbtu',\n", " hist_start=start, hist_end=today)\n", "\n", "# JKM-TTF prices\n", "con_df = loop_historical_data(access_token, endpoint='contracts', contract='jkm-ttf', unit='usd-per-mmbtu', hist_start=start, hist_end=today)\n", "\n", "sab_df.head(5)\n" ] }, { "cell_type": "code", "execution_count": null, "id": "8301f521", "metadata": {}, "outputs": [], "source": [ "# Combining the contract data & the breakevens data into a single dataframe\n", "\n", "# listing all breakevens Dataframes in a list\n", "break_dataframes = [sab_df, bon_df, sab0_df]\n", "break_names = ['Sabine Pass', 'Bonny LNG', 'Sabine Pass Sunk Freight']\n", "\n", "# initialising dataframe with the contract data\n", "combined_df = con_df.copy()\n", "combined_df = combined_df.rename(columns={'Value':'ContractValue'})\n", "\n", "# merging with the breakevens dataframes\n", "dc = 0\n", "for df in break_dataframes:\n", " df = df.rename(columns={'Value':break_names[dc]})\n", " combined_df = pd.merge(combined_df, df[['AsOf', 'PeriodName', break_names[dc]]], on=['AsOf', 'PeriodName'], how='left')\n", " dc+=1\n", "\n", "# defining \"ReleaseDay\" so that data can be filtered by day if needed\n", "combined_df['ReleaseDay'] = combined_df['AsOf'].dt.strftime('%Y-%m-%d')" ] }, { "cell_type": "code", "execution_count": null, "id": "b90cd733", "metadata": {}, "outputs": [], "source": [ "combined_df" ] }, { "cell_type": "markdown", "id": "ec42bfbf", "metadata": {}, "source": [ "# 6. Plotting" ] }, { "cell_type": "code", "execution_count": null, "id": "ac0646ea", "metadata": {}, "outputs": [], "source": [ "# define which month contract to look at\n", "month = 'Jun26'\n", "\n", "# create dataframe by filtering combined_df\n", "month_df = combined_df[combined_df['PeriodName'] == month].copy()\n", "\n", "# creating columns to show the difference between the Contract price and the Breakeven levels\n", "month_df['Sabine Pass - Breakeven Diff'] = month_df['ContractValue'] - month_df['Sabine Pass']\n", "month_df['Bonny LNG - Breakeven Diff'] = month_df['ContractValue'] - month_df['Bonny LNG']\n", "\n", "month_df" ] }, { "cell_type": "code", "execution_count": null, "id": "2805f55e", "metadata": {}, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "import seaborn as sns\n", "\n", "sns.set_style('whitegrid')\n", "\n", "fig, ax = plt.subplots(figsize=(16,6))\n", "\n", "ax.hlines(0, month_df['AsOf'].iloc[-1] + pd.Timedelta(10, unit='day'), month_df['AsOf'].iloc[0] - pd.Timedelta(10, unit='day'), color='grey', alpha=0.85, zorder=0)\n", "\n", "ax.plot(month_df['AsOf'], month_df['ContractValue'], color='#dc853f', label = month + ' JKM-TTF')\n", "ax.plot(month_df['AsOf'], month_df['Sabine Pass'], color='#008bcc', label='USGC Arb Breakeven')\n", "ax.plot(month_df['AsOf'], month_df['Bonny LNG'], color='#008e74', label='Nigeria Arb Breakeven')\n", "#ax.plot(month_df['AsOf'], month_df['Sabine Pass Sunk Freight'], color='#c4242d', label='USGC (Sunk Freight) Arb Breakeven')\n", "\n", "# shading green when the US arb is open (JKM-TTF > Sabine Pass Breakeven)\n", "ax.fill_between(month_df['AsOf'], month_df['ContractValue'], month_df['Sabine Pass'], where=(month_df['Sabine Pass - Breakeven Diff'] > 0), color='mediumseagreen', alpha=0.2)\n", "\n", "# shading red when the Nigeria Arb is closed (JKM-TTF < Bonny LNG Breakeven)\n", "ax.fill_between(month_df['AsOf'], month_df['Bonny LNG'], month_df['ContractValue'], where=(month_df['Bonny LNG - Breakeven Diff'] < 0), color='red', alpha=0.1)\n", "\n", "plt.xlim([month_df['AsOf'].iloc[0]-pd.Timedelta(10, unit='day'), month_df['AsOf'].iloc[-1]+pd.Timedelta(10, unit='day')])\n", "\n", "plt.legend(loc='upper left')" ] } ], "metadata": { "kernelspec": { "display_name": "base", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.5" } }, "nbformat": 4, "nbformat_minor": 5 }