eCommerce API

getShippingNetwork endpoint of API

Overview

        The eCommerce API serves millions of users per year as the back-bone of an online store that generates over $30m in revenue annually. I was the sole architect and coder of this API, building it in Java and deploying through Azure. Some of the functionalities it provides are:

  • Using the customer’s location to determine which warehouses and what inventory should be exposed to them
  • Calculating the shipping fee of an order by using internal algorithms and quoting 3rd party services such as FedEx
  • Managing a “Back in Stock” functionality that emails customers when a product they have subscribed to is back in stock
  • Providing live store information to customers
  • Managing email sendoffs to vendors for automatic product photo collection

        The API features comprehensive logging and a caching system that allows for instantaneous data retrieval and safety from database outages. It was designed with scalability and longevity in mind, using industry-standard practices and thoughtful documentation.

Challenges and Solutions

        The API’s existence was necessitated by the increasingly complex nature of this company’s logistics chain. We previously relied on 3rd party implementations to expose more than one of our warehouses (as the POS did not natively support multi-warehouse inventory), but even that only exposed 2/4 warehouses and came with a whole wealth of problems, and of course the apps cost money. I proposed a new plan to use relational databases to maintain a map of all warehouses with their hierarchical links. For the sake of functionality I divided the facilities into Division (Distribution Center), Warehouse (Store that has a warehouse), and Showrooms. Showrooms are linked to warehouses, and Warehouses are linked to Divisions. If a customer is near a store, we only need to make one request to retrieve the entire warehouse map they can access. I’ll spare the details, but each zipcode is algorithmically mapped to showrooms, and sending that zipcode to the API will result in a response containing all the warehouses and inventory contexts that can be exposed to the customer (along with other metadata, such as localStore, localWarehouse, and localDivision.

        With the inventory covered, the next challenge was serving the customers Shipping Methods and Fees in a timely manner. In order to achieve the fastest results, this endpoint uses threads to quote all relevant shipping carriers at the same time to mitigate time-loss. There is a lot of product analysis that occurs to determine which type of shipping method should be exposed to the customer. The API iterates through the products in the cart and cross-references with its cached product database, checking for things like weight, dimensions, physicality, zonage, and so on. The available results are highly curated due to the sophistication of the company’s supply chain.

        This company did not have a solid contingent plan for maintaining the API if something happened to me or I left the company. I recognized this mistake on their part and sought to minimize technical debt and increase the longevity of the API. To ensure they would be able to manipulate the API without me, I implemented a no-code portal that allows them to change variables, shipping fees, cron-schedules, and copywriting without any knowledge of code or my presence.

No-Code portal of API

Unhappy with a shipping method description? Change it right in here, and the API will update within 5 minutes. Want to refresh inventory more often? Change up the cron-schedule. These are variables and functionalities I felt it important to be able to change quickly. The system has its own validation in place to prevent improper or unreasonable inputs, and will default to a safe hardcoded value coded internally if it detects something wrong. For example, if you miss the documentation that says the shipping fees are in cents and not dollars, the API will reject your input of “99”, which would imply $0.99 shipping. With this system in place I feel confident that leadership and my teams can make the changes they need to in the future, and I am sure future developers will be able to expand it due to its robustness and simplicity.

One of the most sensitive challenges was automatic customer messaging. Features involving customer communication need to have powerful fail-safes that are tested rigorously to ensure that customers cannot be double emailed, misemailed, or in the most grave of cases, spammed infinitely. I was careful to perform all database validation actions before sending the email, and only sending them after validation that the database accepted the update, to ensure that the customer is not sent an email twice erroneously. It feels intuitive to send the message first and then mark it as sent in the database, but this is a dangerous protocol – what happens if the email is sent and then the database or API suffers an outage? The email will never be marked as sent and the system will then try to email it once more.
        Unfortunately there lurked an even greater threat in this company – the ERP was known to very rarely relinquish all reservations, which consequently freed up a ton of inventory. This new inventory would then be exposed on the site, where unsuspecting customers could order the products they think are available, but are actually already reserved by other people. To protect the API from said nightmare, I came up with an algorithm that checks to see if the inventory change is a high enough of a percentage to be considered an anomaly, and in this case no emails will fire off. In case of an anomaly, a human is required to authorize the transmission of emails.

Back in Stock API Functionality

        Finally, with such multi-threaded capability, it was important to make sure our operations were safe. I read the documentation of all objects and methods I use in multi-threaded functions to check if they were thread safe, and in the cases where they are not, I’ve implemented locks. Fortunately the ConcurrentHashMap object is a thread-safe HashMap, which solves most of the issues regarding threads. The StoreList is a single JSONObject that might be retrieved by different users at the same time, and so that object uses a quick lock to allow for sequential reading.

Results and Impact

        The API has been a massive success both from a holistic standpoint and a raw numbers standpoint. Its creation allowed us to expose all four of our distributions centers, of which only two had been exposed prior. That’s over 1 million units (Furniture, no less) that could now be sold online. eCommerce saw a 20% increase in all sales since the implementation of the API. Additionally, the company was able to cut more than $2500 in monthly fees to apps that had been made redundant by the API. The Back In Stock functionality generates about $13k per month at the moment and I expect this to increase as inventory improves and customers become more familiar with the feature. Back in Stock as a functionality existed on our marketing platform, but it did not support multi-warehouse functionality and so we had to turn it off when we made the jump from one to two warehouses. The new system I created is fully dynamic, scalable, and provides a lot of useful analytics on the desires of our customers (I exposed the stats of this system to our Buyers so they can garner a better understanding of what our customers would like to purchase).
        On a holistic level, the control it gave the company was very well received. Previous apps had offered many limitations, and new executive wishes could not even be fulfilled on the old infrastructure. With the API, the sky is now the limit. I have implemented many new functionalities to the API under the auspices of leadership once they realized our new potential. Furthermore, the customer experience has been significantly improved. Long-distance shipping used to cost a flat $500 due to app limitations – now many customers receive lower and more accurate fees such at $330 when they previously would have been charged $500. Customers used to have to call in to have their shipping upgraded to white-glove, whereas now it is a built in option that appears alongside the standard option (and without any time expense due to multi-threading!)