Edited: this post to be the Lemmy.World federation issue post.
We are now ready to upgrade to postgres 16. When this post is 30 mins old the maintenace will start
Updated & Pinned a Comment in this thread explaining my complete investigation and ideas on how the Lemmy app will & could move forward.
Successfully migrated from Postgres 15 to Postgres 16 without issues.
Well done!
Interestingly enough, the number keep growing 😔
https://phiresky.github.io/lemmy-federation-state/site?domain=lemmy.world
Yes... it is VERY annoying. We have so much resources available and lemmy/postgres will not use them
*removed externally hosted image*
That's sad.
I've seen you posted in the Lemmy Matrix channel, hopefully you'll be able to find a way soon. I guess you already read the write-up from Db0? https://dbzer0.com/blog/post-mortem-the-massive-lemmy-world-lemmy-dbzer0-com-federation-delays/
Yes. Unfortunately the information gleamed boils down to two reasons:
Our db server is occasionally slow to respond, but most requests from LW complete in less than 0.1 second. Unfortunately there are times when they take longer. These longer ones are going to be the problem (I believe). As all activities are sequential servers can only catch up as fast as they can process them.
What i've found in the past 30 seconds is that it is not necessarily out database that is the problem but possibly the way lemmy handles the federation. I'm chatting with some of the admins on reddthat and making pretty graphs while looking at walls of logs.
I see, thank you for your work!
I was right https://reddthat.com/comment/8316861