vmaziman@lemm.eetoGeneral Programming Discussion@lemmy.ml•How to send 1 mln requests asyncronously - mechanism?
·
11 months agoMaybe producer consumer?
Producer spits out all the messages to send out onto a message queue, fifo or whatever suits u.
Parrallelizable consumers (think deployed containers) listen to queue and execute request, get response and save it
Scale consumer count up or down as you need to deal with ratelimits
I work for a company that writes software electric and gas’s utility companies use to coordinate power hungry iot devices like thermostats and evs to make power consumption less spikey and more predictable , which lets them supply demand more with solar and wind as opposed to coal peaker plants.
The utility gets a massive savings when peak demand of electricity and gas is controlled more, and pay our company as well as pass a portion of the savings directly to the device owner as a rebate on their utility bill (device owners sign a contract to let utilities manage their devices through us, but every control event has an opt out option so that the device owner can choose not to participate, but the less they opt out the more money they save on the utility bill)