{ "cells": [ { "cell_type": "markdown", "id": "59d19f73", "metadata": {}, "source": [ "# US Arb Freight Breakevens vs Freight Rates - Contract Month Evolution\n", "\n", "This script is used to compare the US Arb Freight Breakevens with Spark30 Freight Rates for a specific contract month.\n", "\n", "For a full explanation of how to import our Arb Breakevens or Spark30 freight data, please refer to our Python Jupyter Notebook Code Samples:\n", "\n", "https://www.sparkcommodities.com/api/code-examples/jupyter.html\n", "\n" ] }, { "cell_type": "markdown", "id": "b0a05be4", "metadata": {}, "source": [ "### Have any questions?\n", "\n", "If you have any questions regarding our API, or need help accessing specific datasets, please contact us at:\n", "\n", "__data@sparkcommodities.com__" ] }, { "cell_type": "markdown", "id": "9e00ae34", "metadata": {}, "source": [ "## 1. Importing Data\n", "\n", "Here we define the functions that allow us to retrieve the valid credentials to access the Spark API.\n", "\n", "__This section can remain unchanged for most Spark API users.__" ] }, { "cell_type": "code", "execution_count": null, "id": "a827bd91", "metadata": {}, "outputs": [], "source": [ "import json\n", "import os\n", "import sys\n", "import pandas as pd\n", "import numpy as np\n", "from base64 import b64encode\n", "from urllib.parse import urljoin\n", "from io import StringIO\n", "import datetime\n", "\n", "try:\n", " from urllib import request, parse\n", " from urllib.error import HTTPError\n", "except ImportError:\n", " raise RuntimeError(\"Python 3 required\")" ] }, { "cell_type": "code", "execution_count": null, "id": "1161e807", "metadata": {}, "outputs": [], "source": [ "API_BASE_URL = \"https://api.sparkcommodities.com\"\n", "\n", "\n", "def retrieve_credentials(file_path=None):\n", " \"\"\"\n", " Find credentials either by reading the client_credentials file or reading\n", " environment variables\n", " \"\"\"\n", " if file_path is None:\n", " client_id = os.getenv(\"SPARK_CLIENT_ID\")\n", " client_secret = os.getenv(\"SPARK_CLIENT_SECRET\")\n", " if not client_id or not client_secret:\n", " raise RuntimeError(\n", " \"SPARK_CLIENT_ID and SPARK_CLIENT_SECRET environment vars required\"\n", " )\n", " else:\n", " # Parse the file\n", " if not os.path.isfile(file_path):\n", " raise RuntimeError(\"The file {} doesn't exist\".format(file_path))\n", "\n", " with open(file_path) as fp:\n", " lines = [l.replace(\"\\n\", \"\") for l in fp.readlines()]\n", "\n", " if lines[0] in (\"clientId,clientSecret\", \"client_id,client_secret\"):\n", " client_id, client_secret = lines[1].split(\",\")\n", " else:\n", " print(\"First line read: '{}'\".format(lines[0]))\n", " raise RuntimeError(\n", " \"The specified file {} doesn't look like to be a Spark API client \"\n", " \"credentials file\".format(file_path)\n", " )\n", "\n", " print(\">>>> Found credentials!\")\n", " \n", " return client_id, client_secret\n", "\n", "\n", "def do_api_post_query(uri, body, headers):\n", " \"\"\"\n", " OAuth2 authentication requires a POST request with client credentials before accessing the API.\n", " This POST request will return an Access Token which will be used for the API GET request.\n", " \"\"\"\n", " url = urljoin(API_BASE_URL, uri)\n", "\n", " data = json.dumps(body).encode(\"utf-8\")\n", "\n", " # HTTP POST request\n", " req = request.Request(url, data=data, headers=headers)\n", " try:\n", " response = request.urlopen(req)\n", " except HTTPError as e:\n", " print(\"HTTP Error: \", e.code)\n", " print(e.read())\n", " sys.exit(1)\n", "\n", " resp_content = response.read()\n", "\n", " # The server must return HTTP 201. Raise an error if this is not the case\n", " assert response.status == 201, resp_content\n", "\n", " # The server returned a JSON response\n", " content = json.loads(resp_content)\n", "\n", " return content\n", "\n", "\n", "def do_api_get_query(uri, access_token, format='json'):\n", " \"\"\"\n", " After receiving an Access Token, we can request information from the API.\n", " Supports both JSON (default) and CSV responses.\n", " \"\"\"\n", " url = urljoin(API_BASE_URL, uri)\n", "\n", " if format == 'json':\n", " headers = {\n", " \"Authorization\": \"Bearer {}\".format(access_token),\n", " \"Accept\": \"application/json\",\n", " }\n", " elif format == 'csv':\n", " headers = {\n", " \"Authorization\": \"Bearer {}\".format(access_token),\n", " \"Accept\": \"text/csv\",\n", " }\n", " else:\n", " raise ValueError(\"The format parameter only takes 'csv' or 'json' as inputs\")\n", "\n", " print(\"Fetching {}\".format(url))\n", "\n", " # HTTP GET request\n", " req = request.Request(url, headers=headers)\n", " try:\n", " response = request.urlopen(req)\n", " except HTTPError as e:\n", " print(\"HTTP Error: \", e.code)\n", " print(e.read())\n", " sys.exit(1)\n", "\n", " resp_content = response.read()\n", "\n", " # The server must return HTTP 200. Raise an error if this is not the case\n", " assert response.status == 200, resp_content\n", "\n", " # Storing response based on requested format\n", " if format == 'json':\n", " content = json.loads(resp_content)\n", " elif format == 'csv':\n", " content = resp_content\n", "\n", " return content\n", "\n", "\n", "def get_access_token(client_id, client_secret):\n", "\n", " payload = \"{}:{}\".format(client_id, client_secret).encode()\n", " headers = {\n", " \"Authorization\": b64encode(payload).decode(),\n", " \"Accept\": \"application/json\",\n", " \"Content-Type\": \"application/json\",\n", " }\n", " body = {\n", " \"grantType\": \"clientCredentials\",\n", " }\n", "\n", " content = do_api_post_query(uri=\"/oauth/token/\", body=body, headers=headers)\n", "\n", " print(\n", " \">>>> Successfully fetched an access token {}****, valid {} seconds.\".format(\n", " content[\"accessToken\"][:5], content[\"expiresIn\"]\n", " )\n", " )\n", "\n", " return content[\"accessToken\"]" ] }, { "cell_type": "markdown", "id": "b46f962b", "metadata": {}, "source": [ "## N.B. Credentials\n", "\n", "Here we call the above functions, and input the file path to our credentials.\n", "\n", "N.B. You must have downloaded your client credentials CSV file before proceeding. Please refer to the API documentation if you have not dowloaded them already. Instructions for downloading your credentials can be found here:\n", "\n", "https://www.sparkcommodities.com/api/request/authentication.html" ] }, { "cell_type": "code", "execution_count": null, "id": "3acdfe86", "metadata": {}, "outputs": [], "source": [ "# Input the path to your client credentials here\n", "client_id, client_secret = retrieve_credentials(file_path=\"/tmp/client_credentials.csv\")\n", "\n", "# Authenticate:\n", "access_token = get_access_token(client_id, client_secret)" ] }, { "cell_type": "markdown", "id": "b0a4c7c9", "metadata": {}, "source": [ "# 2. Calling Freight & Breakevens Data" ] }, { "cell_type": "markdown", "id": "7adcc5ed", "metadata": {}, "source": [ "## 2.1 Fetching Breakevens data\n", "\n", "Here we define the functions used to call the Arb Breakevens data from the Cargo Arb Breakevens API. For more information on the Cargo Arb Breakevens API, please visit the API website or download our Cargo Arb Breakevens code sample:\n", "\n", "https://www.sparkcommodities.com/api/lng-cargo/netbacks-arb-breakevens.html" ] }, { "cell_type": "code", "execution_count": null, "id": "aaefce45", "metadata": {}, "outputs": [], "source": [ "# Define the function for fetching reference data\n", "def list_netbacks(access_token):\n", "\n", " content = do_api_get_query(\n", " uri=\"/v1.0/netbacks/reference-data/\", access_token=access_token\n", " )\n", "\n", " # retrieve release dates separately\n", " reldates = content[\"data\"][\"staticData\"][\"sparkReleases\"]\n", "\n", " # return reference data\n", " refdata_dict = content[\"data\"]\n", "\n", " return reldates, refdata_dict\n", "\n", "# Fetch reference data:\n", "reldates, refdata_dict = list_netbacks(access_token)\n", "\n", "# formatting as a Dataframe\n", "available_df = pd.json_normalize(refdata_dict['staticData']['fobPorts'])\n", "available_df.head(5)" ] }, { "cell_type": "code", "execution_count": null, "id": "eb563eb4", "metadata": {}, "outputs": [], "source": [ "from io import StringIO\n", "\n", "# Defining function to fetch breakevens dataset\n", "\n", "def fetch_breakevens(access_token, ticker, via=None, breakeven='freight', start=None, end=None, format='json'):\n", "\n", " query_params = breakeven + '/' + \"?fob-port={}\".format(ticker)\n", "\n", " if via is not None:\n", " query_params += \"&via-point={}\".format(via)\n", " if start is not None:\n", " query_params += \"&start={}\".format(start)\n", " if end is not None:\n", " query_params += \"&end={}\".format(end)\n", "\n", " uri = \"/v1.0/netbacks/arb-breakevens/{}\".format(query_params)\n", " print(uri)\n", " content = do_api_get_query(\n", " uri=\"/v1.0/netbacks/arb-breakevens/{}\".format(query_params),\n", " access_token=access_token, format=format,\n", " )\n", " \n", " if format == 'json':\n", " my_dict = content['data']\n", " else:\n", " my_dict = content.decode('utf-8')\n", " my_dict = pd.read_csv(StringIO(my_dict))\n", "\n", " return my_dict\n" ] }, { "cell_type": "code", "execution_count": null, "id": "30d317eb", "metadata": {}, "outputs": [], "source": [ "# fetching Sabine Pass UUID from reference data\n", "tsab = available_df[available_df[\"name\"] == 'Sabine Pass'][\"uuid\"].iloc[0]\n", "\n", "# Fetching data in CSV format\n", "break_df = fetch_breakevens(access_token, tsab, via='cogh', breakeven='freight', format='csv')\n", "\n", "break_df['ReleaseDate'] = pd.to_datetime(break_df['ReleaseDate'])\n", "break_df['LoadingDate'] = pd.to_datetime(break_df['LoadingDate'])\n", "\n", "break_df.head(3)" ] }, { "cell_type": "markdown", "id": "de4957ed", "metadata": {}, "source": [ "## 2.2 Fetching Freight Prices\n", "\n", "Here we define the functions used to call the Spark30S Freight data from the Freight Contracts API. For more information on the Freight Contracts API, please visit the API website or download our Freight Contracts code sample:\n", "\n", "https://www.sparkcommodities.com/api/lng-freight/contracts.html" ] }, { "cell_type": "code", "execution_count": null, "id": "08ec1036", "metadata": {}, "outputs": [], "source": [ "# Defining function to fetch Freight dataset\n", "\n", "def fetch_historical_price_releases(access_token, ticker, limit=4, offset=None, vessel=None):\n", "\n", " query_params = \"?limit={}\".format(limit)\n", " if offset is not None:\n", " query_params += \"&offset={}\".format(offset)\n", " \n", " # '174-2stroke' or '160-tfde'\n", " if vessel is not None:\n", " query_params += \"&vessel-type={}\".format(vessel)\n", " \n", " print(\"/v1.0/contracts/{}/price-releases/{}\".format(ticker, query_params))\n", " \n", " content = do_api_get_query(\n", " uri=\"/v1.0/contracts/{}/price-releases/{}\".format(ticker, query_params),\n", " access_token=access_token,\n", " )\n", "\n", " my_dict = content['data']\n", " \n", " return my_dict\n", "\n", "# Defining function to store freight prices as a Pandas DataFrame\n", "\n", "import datetime\n", "def fetch_prices(ticker, limit, my_vessel=None):\n", " my_dict_hist = fetch_historical_price_releases(access_token, ticker, limit=limit, vessel=my_vessel)\n", " \n", " release_dates = []\n", " period_start = []\n", " usd_day = []\n", "\n", " day_min = []\n", " day_max = []\n", " period_index = []\n", "\n", " for release in my_dict_hist:\n", " release_date = release[\"releaseDate\"]\n", "\n", " data_points = release[\"data\"][0][\"dataPoints\"]\n", "\n", " for data_point in data_points:\n", " \n", " release_dates.append(release_date)\n", " \n", " period_start_at = data_point[\"deliveryPeriod\"][\"startAt\"]\n", " period_start.append(period_start_at)\n", "\n", " period_index.append(data_point[\"deliveryPeriod\"][\"name\"])\n", "\n", " usd_day.append(data_point['derivedPrices']['usdPerDay']['spark'])\n", " day_min.append(data_point['derivedPrices']['usdPerDay']['sparkMin'])\n", " day_max.append(data_point['derivedPrices']['usdPerDay']['sparkMax'])\n", "\n", " ## Storing values in a Pandas DataFrame\n", "\n", " historical_df = pd.DataFrame({\n", " 'ReleaseDate': release_dates,\n", " 'Ticker': ticker,\n", " 'PeriodStart': period_start,\n", " 'PeriodIndex': period_index,\n", " 'USDperday': usd_day,\n", " 'USDperdayMax': day_max,\n", " 'USDperdayMin': day_min})\n", "\n", " historical_df['USDperday'] = pd.to_numeric(historical_df['USDperday'])\n", " historical_df['USDperdayMax'] = pd.to_numeric(historical_df['USDperdayMax'])\n", " historical_df['USDperdayMin'] = pd.to_numeric(historical_df['USDperdayMin'])\n", "\n", " historical_df['ReleaseDate'] = pd.to_datetime(historical_df['ReleaseDate'])\n", " historical_df['PeriodStart'] = pd.to_datetime(historical_df['PeriodStart'])\n", " \n", " return historical_df\n" ] }, { "cell_type": "code", "execution_count": null, "id": "efcb7515", "metadata": {}, "outputs": [], "source": [ "# Using functions to fetch freight data\n", "# Here we import Atlantic rates, with the aim to compare against US arb breakevens\n", "\n", "spark30s = fetch_prices(ticker='spark30s', limit=len(break_df['ReleaseDate'].unique()), my_vessel='174-2stroke')\n", "spark30s['PeriodName'] = spark30s['ReleaseDate'].dt.strftime('%b%y')\n", "\n", "spark30ffa = fetch_prices(ticker='spark30ffa-monthly', limit=len(break_df['ReleaseDate'].unique()), my_vessel='174-2stroke')\n", "spark30ffa['PeriodName'] = spark30ffa['PeriodStart'].dt.strftime('%b%y')\n", "\n", "print(\"Columns:\", spark30s.columns.tolist())" ] }, { "cell_type": "code", "execution_count": null, "id": "a0c74063", "metadata": {}, "outputs": [], "source": [ "spark30ffa.head(3)" ] }, { "cell_type": "markdown", "id": "fd359948", "metadata": {}, "source": [ "# 3. Filtering for Fixing Month of Choice" ] }, { "cell_type": "code", "execution_count": null, "id": "9de9bfac", "metadata": {}, "outputs": [], "source": [ "# Choose fixing month of interest here\n", "month = 'Jun26'" ] }, { "cell_type": "code", "execution_count": null, "id": "38b3d759", "metadata": {}, "outputs": [], "source": [ "# Combining Spot and FFA datasets for month of choice\n", "freight_df = pd.concat([\n", " spark30s[spark30s['PeriodName'] == month][['ReleaseDate', 'Ticker', 'PeriodName', 'USDperday', 'USDperdayMax', 'USDperdayMin']],\n", " spark30ffa[(spark30ffa['PeriodName'] == month) & (spark30ffa['PeriodIndex'] != 'M+0')][['ReleaseDate', 'Ticker', 'PeriodName', 'USDperday', 'USDperdayMax', 'USDperdayMin']],\n", "])\n", "\n", "freight_df.head(5)" ] }, { "cell_type": "code", "execution_count": null, "id": "6387392a", "metadata": {}, "outputs": [], "source": [ "# Filtering breakevens data for month of choice\n", "# PeriodName is filtered for Month+1 as breakevens data is organised by load month, not fixing month (assumption: load month = fixing month + 1)\n", "break_df['PeriodName'] = break_df['LoadingDate'].dt.strftime('%b%y')\n", "break_month_str = (datetime.datetime.strptime(month, '%b%y') + datetime.timedelta(days=35)).strftime('%b%y')\n", "\n", "break_month = break_df[break_df['PeriodName'] == break_month_str]\n", "break_month.head(5)" ] }, { "cell_type": "code", "execution_count": null, "id": "b1d5ccf7", "metadata": {}, "outputs": [], "source": [ "# Merge the Freight & Breakevens datasets into one dataframe\n", "\n", "final_df = pd.merge(\n", " freight_df,\n", " break_month[['ReleaseDate', 'FreightBreakeven']],\n", " left_on='ReleaseDate',\n", " right_on = 'ReleaseDate'\n", ")\n", "\n", "final_df = final_df.rename(columns={\n", " 'USDperday': 'Spark30',\n", " 'USDperdayMax': 'Spark30Max',\n", " 'USDperdayMin': 'Spark30Min',\n", "})\n", "\n", "final_df" ] }, { "cell_type": "markdown", "id": "3567b011", "metadata": {}, "source": [ "# 4. Plotting" ] }, { "cell_type": "code", "execution_count": null, "id": "328faa9b", "metadata": {}, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "import seaborn as sns\n", "\n", "sns.set_style(\"whitegrid\")\n", "\n", "fig, ax = plt.subplots(figsize=(15,7))\n", "\n", "# Plot Freight & Breakevens datasets individually\n", "ax.plot(final_df['ReleaseDate'],final_df['Spark30'], color = '#48C38D', linewidth=2.5, label='Spark30S (Atlantic)')\n", "ax.plot(final_df['ReleaseDate'],final_df['FreightBreakeven'], color='#4F41F4', linewidth=2, label='US Arb [M+1] Freight Breakeven Level')\n", "\n", "# Conditional shading to show whether Breakevens are lower/higher than current Freight rates\n", "ax.fill_between(final_df['ReleaseDate'], final_df['Spark30'], final_df['FreightBreakeven'], \\\n", " where=final_df['Spark30']>final_df['FreightBreakeven'], facecolor='red', interpolate=True, alpha=0.05)\n", "\n", "ax.fill_between(final_df['ReleaseDate'], final_df['Spark30'], final_df['FreightBreakeven'], \\\n", " where=final_df['Spark30']