Updated 07-24-2021 | 8 min read

Asynchronous update of related items with AWS DataStore in a Next.js web app.

Vercel custom domain dashboard

TLDR

When working with AWS DataStore you have to deal with async/await operations.

Updating a list of items when the order of async operations execution is not mandatory is viable to do with Promise.all() and map.

Updating related items, where the promise result from the previous item is needed as input for the next item, can be achieved with the for await...of statement. Try the working demo.

Introduction

As you can see in the previous picture we have a UI consisting of a table showing several SaleItems. There you can add, delete, update and list SaleItems.

We want to do something like MS Excel, where we have cells that depend on each other (recursively). For example if we update B3, then B4, and B5 should also be updated based on the previous cell value; B4 depends on B5’s value and B5 depends on B4’s value.

Here is a section of the schema.graphql, it includes the relevant types (entities):

type SaleItem
  @model
  @auth(rules: [{ allow: public }])
  @key(name: "bySalesPlanner", fields: ["salesplannerID"]) {
  id: ID!
  quantity: Float
  priceBasedOnRoi: Boolean
  price: Float
  accSold: Float
  accProfit: Float
  sharesRemain: Float
  salesplannerID: ID
}

type SalesPlanner @model @auth(rules: [{ allow: public }]) {
  id: ID!
  SaleItems: [SaleItem] @connection(keyName: "bySalesPlanner", fields: ["id"])
}

In this model I decided to include calculated fields like accSold, accProfit and sharesRemain as part of the SaleItem entity.

The main reason for that decision was that in the current state of computing vs storage, storage is less expensive than computing. DynamoDB was built with that constraint in mind, optimized for reading versus saving space in storage. To take full advantage of that we should use a single table design model, for now we use a relational model with a DynamoDB database under it.

In my use case having the calculated values stored on the database makes it easier to do CRUD operations because we don’t have to calculate them recursively, avoiding recursive operations complexity and improving UX responsiveness for large datasets.

A single table design approach is recommended after we have certainty that the access patterns are not going to change too much. I am in the first iterations of the app so I will stick to the relational approach. Also I will assume that you know how to model it.

In this case we have a one to many relationship between SalesPlanner and SaleItem. That kind of relational modeling is easier using Amplify Studio, where visually you can add Models (entities), relationship and auth rules to your model. It allows sharing your model and managing content like a CMS. You can even use it without an AWS account.

The AWS DataStore api is very straightforward, check the docs for more info.

List SaleItems

The list of items is kept in the state variable salesItems. The item state variable is used to be able to discern between create and delete/update paths in the form’s onSubmit.

const [salesItems, setSalesItems] = useState([]);
const [item, setItem] = useState(null);

In this effect we access the SaleItems related to the current salesplannerID. It serves to have real time access to those items, the subscription is waiting for changes in the type SaleItem to fire an update to the list.

useEffect(() => {
  const getItems = async () => {
    const models = await DataStore.query(SaleItem, (si) =>
      si.salesplannerID("eq", salesplannerID)
    );
    setSalesItems(models);
  };

  const subscription = DataStore.observe(SaleItem).subscribe((msg) => {
    getItems();
  });

  getItems();

  return () => subscription.unsubscribe();
}, [salesplannerID]);

The types are created when you use the CLI command amplify codegen models. Then

import { SaleItem } from "../src/models";

Add SaleItem

Here the main point is to consider the case when there is a previous element in the list because it is used to calculate accSold, accProfit and sharesRemain. The new item is added to the end of the list like in a FIFO data structure, so only the previous item is relevant.

salesItems is a state variable. The data param is passed from the form created with react-hook-form api.

const prevItem =
  salesItems.length > 0 ? salesItems[salesItems.length - 1] : null;

await DataStore.save(
  new SaleItem({
    "quantity": data.quantity,
    "priceBasedOnRoi": true,
    "price": data.price,
    "accSold": AccSold(prevItem, data.price, data.quantity),
    "accProfit": AccProfit(
      prevItem,
      data.price,
      data.quantity,
      totalInvestment
    ),
    "sharesRemain": SharesRemain(prevItem, shares, data.quantity),
    "salesplannerID": salesplannerID,
  })
);

Delete SaleItem.

The list (table) has access to each item that is going to be deleted. We can pass it to the respective onClick event.

 onClick={() => handleDeleteItem(item, index)}

The handleDeleteItem function deletes the item but if it is not the last one updates the rest based on it.

Operations on DataStore models are asynchronous.

const handleDeleteItem = async (item, index) => {
  const { id } = item;
  const modelToDelete = await DataStore.query(SaleItem, id);
  DataStore.delete(modelToDelete);

  if (index !== salesItems.length - 1) {
    await updateSaleItemsOnDelete(item, index);
  }
};

The updateSaleItemsOnDelete function is more complex.

Here we get from the state salesItems only the slice of items that we have to update.

const saleItemsToUpdate = salesItems.slice(index + 1);

So we need to do async operations on a list but when you delete an item, it behaves like a constant cause we are going to add or subtract the same amount to the calculated fields, we get rid of the items related constraint for the async operations. This means that there is no need to wait for the current operation to finish to do the next.

Also it is helpful because we don’t have a guarantee that DataStore would return the values in the order specified in the map iterations. Some take more time than others, even when all are accessed in the local store in the browser.

DataSore tries to access the data locally first, if not found it connects to the database. The updated data is synced in the background. The DataStore sync explanation and conflict detection / resolution strategies are outside of the scope of this post.

In this scenario Promise.all(), serves your purpose. It executes in parallel (almost) the iterable of promises passed as input and returns a Promise when all the promises have resolved or rejects immediately when any of the input promises rejects.

We are not going to use the returned promise in this case, we just need to guarantee that all fail if any of the async operations fails.

const updateSaleItemsOnDelete = async (toDelete, index) => {
  const saleItemsToUpdate = salesItems.slice(index + 1);

  await Promise.all(
    saleItemsToUpdate.map(async (toUpdate) => {
      let profit = toUpdate.accProfit - toDelete.price * toDelete.quantity;
      let sold = toUpdate.accSold - toDelete.price * toDelete.quantity;
      let remain = toUpdate.sharesRemain + toDelete.quantity;

      await DataStore.save(
        SaleItem.copyOf(toUpdate, (updated) => {
          // Update the values on {updated} variable to update DataStore entry
          updated.accSold = parseFloat(sold.toFixed(4));
          updated.accProfit = parseFloat(profit.toFixed(4));
          updated.sharesRemain = parseFloat(remain.toFixed(4));
        })
      );
    })
  );
};

DataStore has a special notation to update items because models in DataStore are immutable. Use the copyOf function to update the item’s field rather than the mutation of the instance directly.

SaleItem.copyOf(toUpdate, (updated) => {
  // Update the values on {updated} variable to update DataStore entry
  updated.accSold = parseFloat(sold.toFixed(4));
  updated.accProfit = parseFloat(profit.toFixed(4));
  updated.sharesRemain = parseFloat(remain.toFixed(4));
});

Update SaleItems

With the update operation (when we hit the update button) we have a different situation. The next item depends on the previous item, in consequence the order must be respected.

If we try to use Promise.all with map we could encounter in the situation that the prev item is not updated in time before the next iteration.

To guarantee that we have access to the updated prev, we use for await of, it iterates over async iterables objects, waiting to complete the previous pending operations before continuing to the next iteration.

const updateSaleItemsOnUpdate = async (data) => {
  const index = salesItems.findIndex((s) => s.id === item?.id);
  const prevItem = salesItems.length > 0 ? salesItems[index - 1] : null;
  const saleItemsToUpdate = salesItems.slice(index + 1);

  // update selected item
  let prev = await DataStore.save(
    SaleItem.copyOf(item, (u) => {
      u.price = data.price;
      u.quantity = data.quantity;
      u.accSold = AccSold(prevItem, data.price, data.quantity);
      u.accProfit = AccProfit(
        prevItem,
        data.price,
        data.quantity,
        totalInvestment
      );
      u.sharesRemain = SharesRemain(prevItem, shares, data.quantity);
    })
  );

  // update the remaining items base on the selected item update

  for await (const toUpdate of saleItemsToUpdate) {
    prev = await DataStore.save(
      SaleItem.copyOf(toUpdate, (item) => {
        // Update the values on {item} variable to update DataStore entry
        item.accSold = AccSold(prev, toUpdate.price, toUpdate.quantity);
        item.accProfit = AccProfit(
          prev,
          toUpdate.price,
          toUpdate.quantity,
          totalInvestment
        );
        item.sharesRemain = SharesRemain(prev, shares, toUpdate.quantity);
      })
    );
  }
};

Keep me in the loop!

Created by Yampier Medina

© 2020 MIT Licence