{ "cells": [ { "cell_type": "markdown", "id": "59d19f73", "metadata": {}, "source": [ "# Arb Freight Breakevens vs Spot Freight Rates\n", "\n", "This script is used to compare several Arb Freight Breakevens with Spark30S Spot Freight Rates.\n", "\n", "For a full explanation of how to import our Arb Breakevens or Spark30S data, please refer to our Python Jupyter Notebook Code Samples:\n", "\n", "https://www.sparkcommodities.com/api/code-examples/jupyter.html\n", "\n" ] }, { "cell_type": "markdown", "id": "b0a05be4", "metadata": {}, "source": [ "### Have any questions?\n", "\n", "If you have any questions regarding our API, or need help accessing specific datasets, please contact us at:\n", "\n", "__data@sparkcommodities.com__" ] }, { "cell_type": "markdown", "id": "9e00ae34", "metadata": {}, "source": [ "## 1. Importing Data\n", "\n", "Here we define the functions that allow us to retrieve the valid credentials to access the Spark API.\n", "\n", "__This section can remain unchanged for most Spark API users.__" ] }, { "cell_type": "code", "execution_count": null, "id": "57bc3daf", "metadata": {}, "outputs": [], "source": [ "import json\n", "import os\n", "import sys\n", "import pandas as pd\n", "import numpy as np\n", "from base64 import b64encode\n", "from urllib.parse import urljoin\n", "from io import StringIO\n", "import datetime\n", "\n", "try:\n", " from urllib import request, parse\n", " from urllib.error import HTTPError\n", "except ImportError:\n", " raise RuntimeError(\"Python 3 required\")" ] }, { "cell_type": "code", "execution_count": null, "id": "1161e807", "metadata": {}, "outputs": [], "source": [ "API_BASE_URL = \"https://api.sparkcommodities.com\"\n", "\n", "\n", "def retrieve_credentials(file_path=None):\n", " \"\"\"\n", " Find credentials either by reading the client_credentials file or reading\n", " environment variables\n", " \"\"\"\n", " if file_path is None:\n", " client_id = os.getenv(\"SPARK_CLIENT_ID\")\n", " client_secret = os.getenv(\"SPARK_CLIENT_SECRET\")\n", " if not client_id or not client_secret:\n", " raise RuntimeError(\n", " \"SPARK_CLIENT_ID and SPARK_CLIENT_SECRET environment vars required\"\n", " )\n", " else:\n", " # Parse the file\n", " if not os.path.isfile(file_path):\n", " raise RuntimeError(\"The file {} doesn't exist\".format(file_path))\n", "\n", " with open(file_path) as fp:\n", " lines = [l.replace(\"\\n\", \"\") for l in fp.readlines()]\n", "\n", " if lines[0] in (\"clientId,clientSecret\", \"client_id,client_secret\"):\n", " client_id, client_secret = lines[1].split(\",\")\n", " else:\n", " print(\"First line read: '{}'\".format(lines[0]))\n", " raise RuntimeError(\n", " \"The specified file {} doesn't look like to be a Spark API client \"\n", " \"credentials file\".format(file_path)\n", " )\n", "\n", " print(\">>>> Found credentials!\")\n", " \n", " return client_id, client_secret\n", "\n", "\n", "def do_api_post_query(uri, body, headers):\n", " \"\"\"\n", " OAuth2 authentication requires a POST request with client credentials before accessing the API.\n", " This POST request will return an Access Token which will be used for the API GET request.\n", " \"\"\"\n", " url = urljoin(API_BASE_URL, uri)\n", "\n", " data = json.dumps(body).encode(\"utf-8\")\n", "\n", " # HTTP POST request\n", " req = request.Request(url, data=data, headers=headers)\n", " try:\n", " response = request.urlopen(req)\n", " except HTTPError as e:\n", " print(\"HTTP Error: \", e.code)\n", " print(e.read())\n", " sys.exit(1)\n", "\n", " resp_content = response.read()\n", "\n", " # The server must return HTTP 201. Raise an error if this is not the case\n", " assert response.status == 201, resp_content\n", "\n", " # The server returned a JSON response\n", " content = json.loads(resp_content)\n", "\n", " return content\n", "\n", "\n", "def do_api_get_query(uri, access_token, format='json'):\n", " \"\"\"\n", " After receiving an Access Token, we can request information from the API.\n", " Supports both JSON (default) and CSV responses.\n", " \"\"\"\n", " url = urljoin(API_BASE_URL, uri)\n", "\n", " if format == 'json':\n", " headers = {\n", " \"Authorization\": \"Bearer {}\".format(access_token),\n", " \"Accept\": \"application/json\",\n", " }\n", " elif format == 'csv':\n", " headers = {\n", " \"Authorization\": \"Bearer {}\".format(access_token),\n", " \"Accept\": \"text/csv\",\n", " }\n", " else:\n", " raise ValueError(\"The format parameter only takes 'csv' or 'json' as inputs\")\n", "\n", " print(\"Fetching {}\".format(url))\n", "\n", " # HTTP GET request\n", " req = request.Request(url, headers=headers)\n", " try:\n", " response = request.urlopen(req)\n", " except HTTPError as e:\n", " print(\"HTTP Error: \", e.code)\n", " print(e.read())\n", " sys.exit(1)\n", "\n", " resp_content = response.read()\n", "\n", " # The server must return HTTP 200. Raise an error if this is not the case\n", " assert response.status == 200, resp_content\n", "\n", " # Storing response based on requested format\n", " if format == 'json':\n", " content = json.loads(resp_content)\n", " elif format == 'csv':\n", " content = resp_content\n", "\n", " return content\n", "\n", "\n", "def get_access_token(client_id, client_secret):\n", "\n", " payload = \"{}:{}\".format(client_id, client_secret).encode()\n", " headers = {\n", " \"Authorization\": b64encode(payload).decode(),\n", " \"Accept\": \"application/json\",\n", " \"Content-Type\": \"application/json\",\n", " }\n", " body = {\n", " \"grantType\": \"clientCredentials\",\n", " }\n", "\n", " content = do_api_post_query(uri=\"/oauth/token/\", body=body, headers=headers)\n", "\n", " print(\n", " \">>>> Successfully fetched an access token {}****, valid {} seconds.\".format(\n", " content[\"accessToken\"][:5], content[\"expiresIn\"]\n", " )\n", " )\n", "\n", " return content[\"accessToken\"]" ] }, { "cell_type": "markdown", "id": "b46f962b", "metadata": {}, "source": [ "## N.B. Credentials\n", "\n", "Here we call the above functions, and input the file path to our credentials.\n", "\n", "N.B. You must have downloaded your client credentials CSV file before proceeding. Please refer to the API documentation if you have not dowloaded them already. Instructions for downloading your credentials can be found here:\n", "\n", "https://www.sparkcommodities.com/api/introduction.html" ] }, { "cell_type": "code", "execution_count": null, "id": "3acdfe86", "metadata": {}, "outputs": [], "source": [ "# Input the path to your client credentials here\n", "client_id, client_secret = retrieve_credentials(file_path=\"/tmp/client_credentials.csv\")\n", "\n", "\n", "# Authenticate:\n", "access_token = get_access_token(client_id, client_secret)" ] }, { "cell_type": "markdown", "id": "7adcc5ed", "metadata": {}, "source": [ "# 2. Fetching Netbacks Arb Breakevens Data" ] }, { "cell_type": "code", "execution_count": null, "id": "aaefce45", "metadata": {}, "outputs": [], "source": [ "# Define the function for listing all netback reference data\n", "def list_netbacks(access_token):\n", "\n", " content = do_api_get_query(\n", " uri=\"/v1.0/netbacks/reference-data/\", access_token=access_token\n", " )\n", "\n", " # retrieve release dates separately\n", " reldates = content[\"data\"][\"staticData\"][\"sparkReleases\"]\n", "\n", " # return reference data\n", " refdata_dict = content[\"data\"]\n", "\n", " return reldates, refdata_dict\n", "\n", "# Fetch reference data:\n", "reldates, refdata_dict = list_netbacks(access_token)\n", "\n", "# Define the function for fetching breakevens data\n", "def fetch_breakevens(access_token, fob_port_uuid, via_point=None, breakeven='freight', start=None, end=None, format='json'):\n", "\n", "\n", " query_params = breakeven + '/' + \"?fob-port={}\".format(fob_port_uuid)\n", "\n", " if via_point is not None:\n", " query_params += \"&via-point={}\".format(via_point)\n", " if start is not None:\n", " query_params += \"&start={}\".format(start)\n", " if end is not None:\n", " query_params += \"&end={}\".format(end)\n", "\n", " uri = \"/v1.0/netbacks/arb-breakevens/{}\".format(query_params)\n", " print(uri)\n", " content = do_api_get_query(\n", " uri=\"/v1.0/netbacks/arb-breakevens/{}\".format(query_params),\n", " access_token=access_token, format=format,\n", " )\n", " \n", " if format == 'json':\n", " my_dict = content['data']\n", " else:\n", " my_dict = content.decode('utf-8')\n", " my_dict = pd.read_csv(StringIO(my_dict))\n", "\n", " return my_dict\n", "\n", "\n", "# Calling reference data and formatting as a Dataframe\n", "available_df = pd.json_normalize(refdata_dict['staticData']['fobPorts'])\n", "available_df.head(3)" ] }, { "cell_type": "markdown", "id": "f28b8e1a", "metadata": {}, "source": [ "### Fetching Breakevens for several FOB Ports\n", "\n", "The function below uses the above functions to call breakevens data for your chosen selection of FOB ports and via points. These datasets are then saved as a single, convenient dataframe." ] }, { "cell_type": "code", "execution_count": null, "id": "d8dcb8e4", "metadata": {}, "outputs": [], "source": [ "def bulk_freight_breakevens(fobs, vias):\n", "\n", " key_cols = ['ReleaseDate', 'LoadMonthIndex', 'LoadMonthStart', 'LoadingDate']\n", "\n", " if len(fobs) != len(vias):\n", " raise ValueError(\n", " f\"`fobs` and `vias` must be the same length \"\n", " f\"(got {len(fobs)} fobs, {len(vias)} vias).\"\n", " )\n", "\n", " # initialise dataframe\n", " break_df = None\n", " used_names = set()\n", "\n", " # iterate over all FOBs and corresponding via points\n", " for fob, via in zip(fobs, vias):\n", " # get FOB port UUID\n", " matches = available_df.loc[available_df['name'] == fob, 'uuid']\n", " if matches.empty:\n", " raise ValueError(f\"FOB port '{fob}' not found in available_df['name'].\")\n", " fob_uuid = matches.iloc[0]\n", "\n", " # call data\n", " breakeven = fetch_breakevens(\n", " access_token, fob_uuid, via_point=via, breakeven='freight', format='csv'\n", " )\n", " breakeven['ReleaseDate'] = pd.to_datetime(breakeven['ReleaseDate'])\n", "\n", " # naming column as FOB port name\n", " col_name = fob\n", " if col_name in used_names:\n", " col_name = f\"{fob} ({via})\"\n", " if col_name in used_names:\n", " raise ValueError(\n", " f\"Duplicate column '{col_name}' — same fob/via pair passed twice.\"\n", " )\n", " used_names.add(col_name)\n", "\n", " # format current dataframe\n", " this_df = (\n", " breakeven[key_cols + ['FreightBreakeven']]\n", " .rename(columns={'FreightBreakeven': col_name})\n", " .drop_duplicates(subset=key_cols)\n", " )\n", "\n", " # Initialise on the first fob, else outer-merge onto break_df\n", " if break_df is None:\n", " break_df = this_df\n", " else:\n", " break_df = pd.merge(break_df, this_df, on=key_cols, how='outer')\n", "\n", " break_df['ReleaseDate'] = pd.to_datetime(break_df['ReleaseDate'])\n", " break_df['LoadingDate'] = pd.to_datetime(break_df['LoadingDate'])\n", "\n", " break_df = (\n", " break_df.sort_values(['ReleaseDate', 'LoadingDate']).reset_index(drop=True)\n", " )\n", "\n", " used_names = list(used_names)\n", "\n", " return break_df, used_names" ] }, { "cell_type": "code", "execution_count": null, "id": "9c98c68e", "metadata": {}, "outputs": [], "source": [ "# Define the list of FOB ports & correlating via points for the arb breakevens required \n", "fobs = ['Sabine Pass', 'Bonny LNG']\n", "vias = ['cogh', 'cogh']\n", "\n", "# Call function\n", "break_df, break_cols = bulk_freight_breakevens(fobs, vias)\n", "break_df.head()" ] }, { "cell_type": "code", "execution_count": null, "id": "7a80e89c", "metadata": {}, "outputs": [], "source": [ "# storing length of dataset to filter Spark30S data\n", "freight_length = len(break_df['ReleaseDate'].unique())" ] }, { "cell_type": "markdown", "id": "de4957ed", "metadata": {}, "source": [ "# 3. Fetching Spot Freight Prices" ] }, { "cell_type": "code", "execution_count": null, "id": "08ec1036", "metadata": {}, "outputs": [], "source": [ "# Defining function to fetch historical freight data from API\n", "def fetch_historical_price_releases(access_token, ticker, limit, offset=None, vessel=None):\n", "\n", " query_params = \"?limit={}\".format(limit)\n", " if offset is not None:\n", " query_params += \"&offset={}\".format(offset)\n", " \n", " if vessel is not None:\n", " query_params += \"&vessel-type={}\".format(vessel)\n", "\n", " content = do_api_get_query(\n", " uri=\"/v1.0/contracts/{}/price-releases/{}\".format(ticker, query_params),\n", " access_token=access_token,\n", " )\n", "\n", " uri=\"/v1.0/contracts/{}/price-releases/{}\".format(ticker, query_params)\n", " print(uri)\n", "\n", " my_dict = content[\"data\"]\n", "\n", " return my_dict\n", "\n", "\n", "# Defining the function for storing and formatting the data into a Pandas DataFrame\n", "def store_and_format(dict_hist):\n", " stored_data = {\n", " \"ReleaseDate\": [],\n", " \"Ticker\": [],\n", " \"PeriodStart\": [],\n", " \"USDperday\": [],\n", " \"USDperdayMax\": [],\n", " \"USDperdayMin\": [],\n", " }\n", "\n", " for release in dict_hist:\n", "\n", " data_points = release[\"data\"][0][\"dataPoints\"]\n", "\n", " for data_point in data_points:\n", " stored_data['Ticker'].append(release[\"contractId\"])\n", " stored_data['ReleaseDate'].append(release[\"releaseDate\"])\n", "\n", " period_start_at = data_point[\"deliveryPeriod\"][\"startAt\"]\n", " stored_data['PeriodStart'].append(period_start_at)\n", "\n", " stored_data['USDperday'].append(data_point[\"derivedPrices\"][\"usdPerDay\"][\"spark\"])\n", " stored_data['USDperdayMin'].append(data_point[\"derivedPrices\"][\"usdPerDay\"][\"sparkMin\"])\n", " stored_data['USDperdayMax'].append(data_point[\"derivedPrices\"][\"usdPerDay\"][\"sparkMax\"])\n", " \n", " historical_df = pd.DataFrame(stored_data)\n", " \n", " historical_df[\"USDperday\"] = pd.to_numeric(historical_df[\"USDperday\"])\n", " historical_df[\"USDperdayMax\"] = pd.to_numeric(historical_df[\"USDperdayMax\"])\n", " historical_df[\"USDperdayMin\"] = pd.to_numeric(historical_df[\"USDperdayMin\"])\n", "\n", " historical_df[\"ReleaseDate\"] = pd.to_datetime(historical_df[\"ReleaseDate\"])\n", " \n", " return historical_df" ] }, { "cell_type": "code", "execution_count": null, "id": "efcb7515", "metadata": {}, "outputs": [], "source": [ "### Calling Spark30S data using above functions\n", "spark30s_raw = fetch_historical_price_releases(access_token, 'spark30s', limit=freight_length, offset=0)\n", "spark30s = store_and_format(spark30s_raw)\n", "\n", "spark30s.head(5)" ] }, { "cell_type": "markdown", "id": "3567b011", "metadata": {}, "source": [ "# 4. Plotting" ] }, { "cell_type": "code", "execution_count": null, "id": "1fd3546a", "metadata": {}, "outputs": [], "source": [ "# fetch front month breakevens only\n", "front_df = break_df[break_df['LoadMonthIndex']==\"M+1\"]\n", "\n", "# Join spot rates and breakevens dataframes\n", "spark30s['ReleaseDate'] = pd.to_datetime(spark30s['ReleaseDate'])\n", "merge_df = pd.merge(spark30s, front_df, on='ReleaseDate', how='inner').drop(columns=['PeriodStart'])\n", "\n", "merge_df" ] }, { "cell_type": "code", "execution_count": null, "id": "6c736ee6", "metadata": {}, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "import seaborn as sns\n", "\n", "sns.set_style(\"whitegrid\")\n", "\n", "fig, ax = plt.subplots(figsize=(15,7))\n", "\n", "ax.plot(front_df['ReleaseDate'],front_df['Sabine Pass'], color='#4F41F4', linewidth=1.5, alpha=0.6, label='US Arb [M+1] Freight Breakeven Level')\n", "ax.plot(front_df['ReleaseDate'],front_df['Bonny LNG'], color='firebrick', linewidth=1.5, alpha=0.6, label='Bonny LNG Arb [M+1] Freight Breakeven Level')\n", "ax.plot(spark30s['ReleaseDate'],spark30s['USDperday'], color = '#48C38D', linewidth=2.5, label='Spark30S (Atlantic)')\n", "\n", "ax.fill_between(merge_df['ReleaseDate'], merge_df['USDperday'], merge_df['Sabine Pass'], \\\n", " where=merge_df['USDperday']merge_df['Bonny LNG'], facecolor='red', interpolate=True, alpha=0.05)\n", "\n", "ax.set_xlim(datetime.datetime.today() - datetime.timedelta(days=380), datetime.datetime.today())\n", "ax.set_ylim(-50000, 320000)\n", "\n", "plt.title('Spark30S (Atlantic) vs. US Arb [M+1] Freight Breakeven Level')\n", "\n", "sns.despine(left=True, bottom=True)\n", "plt.grid(True)\n", "plt.legend()" ] } ], "metadata": { "kernelspec": { "display_name": "base", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.9" } }, "nbformat": 4, "nbformat_minor": 5 }