Streamlined [JavaScript?] solution for displaying and modifying several location-specific purchasing and selling rates

Question

Streamlined [JavaScript?] solution for displaying and modifying several location-specific purchasing and selling rates

No, this is not related to interviews or cryptocurrencies! :)
It is for a non-profit personal web application designed to enhance a game.

This question involves both finance and coding.
I am developing this web app using Vue.js, so I prefer a JavaScript solution. However, an abstract approach or pseudo-code would also be helpful as I can translate it into JavaScript.

Here is the scenario:
Multiple locations (referred to as "stores") buy and sell various items at different prices.
Prices fluctuate randomly, possibly due to supply and demand, but that is not crucial to the problem.
There will be around 100-1000 locations and 10-100 different items.
The majority of locations (estimated 80-90%) will only deal with a few items.
Some locations (about 1-10%) will handle many items, if not all.
Around 50% of locations may only buy or sell, not both.

The objective is to display an extensive list of all locations that have a buying price for an item compared to locations that have a selling price for that same item (similar to permutations/combinations, but with expected gaps).
The final output should be a simple list/array of objects (keyed/value pairs).

A worst-case scenario could involve a paginated list of 99 million objects (1000 stores * (1000 - 1) stores * 100 items); however, there are anticipated gaps in the data, so ideally there would be significantly fewer than 1 million objects (ideally under 100,000; they do not all need to be live in memory and could be loaded from storage).

For example (sorted by Item and Buy Location for clarity):


Item     Buy Location   Buy Price Ratio Sell Price   Sell Location
Item A   Location A     4         0.5   2            Location B
Item A   Location A     4         2.0   8            Location C
Item A   Location B     6         0.66  4            Location A
Item A   Location B     6         1.33  8            Location C
Item A   Location C     10        0.4   4            Location A
Item A   Location C     10        0.2   2            Location B
Item B   Location A     3         0.33  1            Location B
Item B   Location A     3         1.33  4            Location C
Item B   Location B     3         0.66  2            Location A
Item B   Location B     3         1.33  4            Location C
Item B   Location C     5         0.4   2            Location A
Item B   Location C     5         0.2   1            Location B

Each column in the list is sortable, with the default being "Ratio" to display the optimal combinations of buying and selling locations. When prices are updated, Vue.js automatically updates and resorts the list.

Currently, I have inefficient ideas on how to tackle this issue, which involve regenerating the entire list whenever a single price changes.
Since the final output must support arbitrary sorting, direct indexing or access to items is not plausible.
If the price for a specific location's item changes, there isn't a method to pinpoint and update that specific row or cell.

I suspect that my approach is lacking efficiency, and I believe this task has been addressed before in scalable ways.
For instance, creating a straightforward buy/sell price chart across multiple exchanges.
While considering exploring open-source cryptocurrency bot code, this seems more like a finance-related challenge that I wish to comprehend rather than a programming one that I intend to replicate.

How would you go about implementing this?
What kind of dataset(s) would you utilize?

Thank you!

javascript vue.js vuetify.js

Answer 1

Answer №1

In my perspective

The 1 million lines of data may overwhelm users if presented all at once.
While a 1 million array size may not be too large for Javascript (~10MB, as demonstrated in your application here), rendering it all to the DOM element could pose significant challenges.

How would you tackle this issue?

In such a scenario, I would opt for pagination or aggregation (e.g., displaying the first 10 highest ratios for each item).

For instance:

computed: {
  displayItems () {
    // This will get updated when rawItems, currentPage, or pageSize change
    return doFilterOrSort(rawItems).slice(currentPage * pageSize, pageSize)
  }
}

This approach addresses the problem of an excessive number of DOM elements (assuming that updating a 1 million array size is fast enough not to become a bottleneck).

If your data changes frequently, like 10 times in a second, consider throttling or debouncing to reduce rendering frequency.

Example of throttling:

watch: {
  rawItems () {
    if (!ticking) {
      requestAnimationFrame(() => {
        displayItems = doFilterOrSort(rawItems).slice(currentPage * pageSize, pageSize)
        ticking = false
      })
      ticking = true
    }
  }
}

If you wish to implement infinite scrolling, recycle out-of-screen DOM elements. Refer to examples in react-virtualized.

What type of dataset(s) should you use?

Without details on the data format retrieved (initially and with updates), it's challenging to provide a precise answer.

Assuming a fixed number of stores and items with updates per item, I suggest using three arrays:

1. The first array stores buy/sell values for each item in every store. Updates here are easily done in O(1) time complexity.

2. The second array holds ratio objects for combinations of store-item pairs. Updating this incurs O(n⋅n) operations, e.g., updating 'ab' and 'ba' for a set of 3 stores and 1 item.

3. The third array extends the previous one by incorporating sorting or grouping, costing O((n⋅n⋅m) log (n⋅n⋋m)) (a potential bottleneck requiring careful attention based on array structure).

In case these solutions are insufficient, consider aggregating data on the server before transmitting it via web sockets instead of sending everything to the client side.

Answer 2

In my perspective

The 1 million lines of data may overwhelm users if presented all at once.
While a 1 million array size may not be too large for Javascript (~10MB, as demonstrated in your application here), rendering it all to the DOM element could pose significant challenges.

How would you tackle this issue?

In such a scenario, I would opt for pagination or aggregation (e.g., displaying the first 10 highest ratios for each item).

For instance:

computed: {
  displayItems () {
    // This will get updated when rawItems, currentPage, or pageSize change
    return doFilterOrSort(rawItems).slice(currentPage * pageSize, pageSize)
  }
}

This approach addresses the problem of an excessive number of DOM elements (assuming that updating a 1 million array size is fast enough not to become a bottleneck).

If your data changes frequently, like 10 times in a second, consider throttling or debouncing to reduce rendering frequency.

Example of throttling:

watch: {
  rawItems () {
    if (!ticking) {
      requestAnimationFrame(() => {
        displayItems = doFilterOrSort(rawItems).slice(currentPage * pageSize, pageSize)
        ticking = false
      })
      ticking = true
    }
  }
}

If you wish to implement infinite scrolling, recycle out-of-screen DOM elements. Refer to examples in react-virtualized.

What type of dataset(s) should you use?

Without details on the data format retrieved (initially and with updates), it's challenging to provide a precise answer.

Assuming a fixed number of stores and items with updates per item, I suggest using three arrays:

1. The first array stores buy/sell values for each item in every store. Updates here are easily done in O(1) time complexity.

2. The second array holds ratio objects for combinations of store-item pairs. Updating this incurs O(n⋅n) operations, e.g., updating 'ab' and 'ba' for a set of 3 stores and 1 item.

3. The third array extends the previous one by incorporating sorting or grouping, costing O((n⋅n⋅m) log (n⋅n⋋m)) (a potential bottleneck requiring careful attention based on array structure).

In case these solutions are insufficient, consider aggregating data on the server before transmitting it via web sockets instead of sending everything to the client side.

Streamlined [JavaScript?] solution for displaying and modifying several location-specific purchasing and selling rates

Answer №1

Similar questions

Exploring the world of Vue and Pinia: managing and accessing data like

The synergy between Tailwind's divide-y utility and Vue3's v-for directive

Can the caller function's arguments be altered using Function.prototype.apply()?

Configure Laravel application to use a specific language locale in the app.php file

Trouble arises when attempting to establish an isolated scope within Angular alongside UI Bootstrap

Tips on locating information within a pre-existing GET array with parameters provided

The perfect way to override jest mocks that are specified in the "mocks" directory

What is the optimal method for transmitting both an image and JSON data to my express server?

What is the process for obtaining the hash of every hyperlink?

Upon receiving the ajax request, the entire code page is called back

Compatible with pure vanilla JavaScript, but not jQuery

Updating a boolean prop does not cause the child component to be refreshed

What is the proper way to display the date and time 2021-11-14T18:30:00.000+00:00?

Maximizing the power of datatables with ajax integration

Steps for correctly retrieving a specific product from my API using Vue3

The strict-origin-when-cross-origin policy is enforced when submitting a POST request through a form to a specific route

Duplicate the identical data retrieval query, this time utilizing a JavaScript Ajax post method

React component stuck in endless loop due to Intersection Observer

How to implement a cyclic item generation feature in React.js

Accessing a Child Component Function in Material UI

Streamlined [JavaScript?] solution for displaying and modifying several location-specific purchasing and selling rates

Answer №1

Similar questions

Exploring the world of Vue and Pinia: managing and accessing data like

The synergy between Tailwind's divide-y utility and Vue3's v-for directive

Can the caller function's arguments be altered using Function.prototype.apply()?

Configure Laravel application to use a specific language locale in the app.php file

Trouble arises when attempting to establish an isolated scope within Angular alongside UI Bootstrap

Tips on locating information within a pre-existing GET array with parameters provided

The perfect way to override jest mocks that are specified in the "__mocks__" directory

What is the optimal method for transmitting both an image and JSON data to my express server?

What is the process for obtaining the hash of every hyperlink?

Upon receiving the ajax request, the entire code page is called back

Compatible with pure vanilla JavaScript, but not jQuery

Updating a boolean prop does not cause the child component to be refreshed

What is the proper way to display the date and time 2021-11-14T18:30:00.000+00:00?

Maximizing the power of datatables with ajax integration

Steps for correctly retrieving a specific product from my API using Vue3

The strict-origin-when-cross-origin policy is enforced when submitting a POST request through a form to a specific route

Duplicate the identical data retrieval query, this time utilizing a JavaScript Ajax post method

React component stuck in endless loop due to Intersection Observer

How to implement a cyclic item generation feature in React.js

Accessing a Child Component Function in Material UI

The perfect way to override jest mocks that are specified in the "mocks" directory