I'm editing a story right now, and they use this construction with no speaking verbs a lot:
Rachel was finishing washing the dinner plates, “I know this will be hard on them, but Shelby needs to go on home."
I know you could replace that comma with a period, but I wonder if it's common to use a comma or semicolon to avoid slowing the reader too much?
This next one makes more sense to me because laughing could conceivably be a speaking verb:
Maddie's laugh was laced with sarcasm, “Mark, looks like you’ve got a friend.”
That one could even be a colon...
Am I overthinking this? Should I just replace them all with periods?
As long as they create self-contained sentences, periods/full stops are always a safe option, so you shouldn't need to worry about using them in either of these instances.
Arguably the first comma example should be replaced. As it is, it's splicing two unconnected ideas. A semicolon may seem 'speedier' but I would (personally) avoid even that here since the two parts don't seem (to me, absent context) to be sufficiently tied together to validate that choice. In any case, if this is text for children, the punctuation should be simple, i.e., use a period. If the text is for adults, they shouldn't be assumed to be two-mississippi-ing after periods and, consequently, periods should not be seen as slowing them down. So... just use a period.
In the second example, if not a safe period, a colon is perfectly fine: the dialogue is directly related to the preceding text. A copy edit along the lines of:—
would justify the comma according to the style I've acquired, but that's a much bigger change for the sake of a comma.
It all comes down to taste though, so take my comments with a pinch of salt.