Okay, Tartube can definitely handle what you want with a few additional flags! Here's the mediainfo for the output file after doing a test run on this MrBeast video (note that the audio track is incorrectly marked as English but is indeed Japanese, and that S_TEXT is how SRT appears in an MKV file):

General
Unique ID                                : 242275721910232180380466434100717751726 (0xB6449B54C970D7DBA0EB469BBD590DAE)
Complete name                            : C:\Users\WDAGUtilityAccount\Tartube\Test Audio Playlist\$1 vs $100,000,000 House!.mkv
Format                                   : Matroska
Format version                           : Version 4
File size                                : 556 MiB
Duration                                 : 17 min 35 s
Overall bit rate                         : 4 418 kb/s
Frame rate                               : 29.970 FPS
Writing application                      : Lavf60.3.100
Writing library                          : Lavf60.3.100
ErrorDetectionType                       : Per level 1

Video
ID                                       : 1
Format                                   : VP9
Format profile                           : 0
Codec ID                                 : V_VP9
Duration                                 : 17 min 35 s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 29.970 (29970/1000) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Title                                    : ISO Media file produced by Google Inc. Created on: 11/09/2023.
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709
VENDOR_ID                                : [0][0][0][0]

Audio
ID                                       : 2
Format                                   : Opus
Codec ID                                 : A_OPUS
Duration                                 : 17 min 35 s
Channel(s)                               : 2 channels
Channel layout                           : L R
Sampling rate                            : 48.0 kHz
Bit depth                                : 32 bits
Compression mode                         : Lossy
Delay relative to video                  : 7 ms
Language                                 : English
Default                                  : Yes
Forced                                   : No

Text
ID                                       : 3
Format                                   : UTF-8
Codec ID                                 : S_TEXT/UTF8
Codec ID/Info                            : UTF-8 Plain Text
Duration                                 : 17 min 30 s
Language                                 : English
Default                                  : No
Forced                                   : No

I did this all on Windows 10 in Windows Sandbox with a fresh Tartube install to make sure I didn't have some lurking non-default setting causing unexplained behavior. Here's what to do to get the same results, along with a screen recording of the same process with some rough edits (don't be scared off by the long instructions--it's mostly me just explaining what the options do, and it should only take about five to ten minutes!):

1. Grab the 64-bit Windows installer here

2. Go through the install process leaving everything as default and installing yt-dlp and FFmpeg when prompted

3. Go through the tutorial just to get a sense of how things are laid out (it's a lot to take in so don't expect to remember everything, and I'm going to guide you through the exact steps do don't worry)

4. Add your channel by first copying the /videos URL (e.g. https://youtube.com/@MrBeast/videos), clicking the Add channel icon (second from the left in the toolbar), and entering the channel name (this will be the name of the folder that videos are stored in). If the URL isn't automatically grabbed, paste it into the second box.

5. Right-click the channel in the left-hand menu and select Downloads -> Apply download options

6. Make sure Create new download options is selected and click OK

7. (Optional) Give the options a sensible name, e.g. "Japanese Audio with embedded English SRT"

8. Paste these options into the Additional download options box:

--format bv*+ba[language=ja]/bv*+ba[language=en]/bv*+ba/best
--convert-subs srt
--compat-options no-keep-subs

Explanation of the options:
--convert-subs is pretty self explanatory--it will convert the YouTube VTT subs to SRT.
--format: the format selection is a hierarchy delineated by the /. First, it tries to download the best video with the best audio in Japanese (bv*+ba[language=ja]). If Japanese audio isn't present, it tries English audio (bv*+ba[language=en]). If neither are present (which can also happen if the uploader failed to mark the language correctly), it grabs whatever the default audio track is. If all else fails, it grabs the best combined format (this should realistically never happen on YouTube). If you dislike any of those fallback options and/or would prefer that the download simply fail, feel free to delete any/all of them along with the preceding /, although I recommend at least leaving bv*+ba. For your use case, --format bv*+ba[language=ja] is the bare minimum which will fail if there isn't an audio track explicitly labeled as Japanese.)
--compat-options no-keep-subs is necessary to make sure the subtitles are deleted after merging them into the MKV, since the options we will be setting through the GUI include both --write-subs and --embed-subs, and the default behavior in this scenario is to both embed the subs and write them to an external file. If you prefer to keep the external SRT file, simply remove this line.

9. (Optional) Click the Files tab at the top and customize the filename format. Personally, I'm partial to
   %(upload_date)s %(title)s-%(id)s.%(ext)s
   so that I can naturally sort things by upload date and easily go between URLs and videos (since YouTube URLs are just https://youtube.com/watch?v=[id]), but if you're happy with the default title-only you can leave this be.

10. Click the Formats tab at the top. Set the drop-down for If merge is required after post-processing, output this format: to mkv. It will give you a warning that you need to also add it above, but as far as I can tell this is neither true (works fine without it) nor possible (mkv isn't even listed there). If you do prefer specific video/audio formats or want a specific/maximum resolution, let me know and I can change the format option to accommodate that preference, since unfortunately this tab doesn't account for multiple audio tracks.

11. Click the Subtitles tab at the top. Ensure that Download subtitle file for these languages: is selected and that English [en] is listed (if your default Windows language is English I think it'll already be there, but if not, add it from the list on the left). Note that this will not grab the automatically-generated subtitles from YouTube, but it sounds like you don't need these for your specific situation.

12. Click the More options sub-tab. Under Preferred subtitle formats write srt/best (I honestly don't think this will affect YouTube since all subs seem to be VTT, but it can't hurt). More importantly, check the box for During post-processing, merge subtitles file with video.

13. Click OK in the lower-right to save the download options. You're done with the setup!

If you want to download the entire channel in this way, right-click the channel in the left-hand menu and click Download channel. You can monitor the download progress in the Progress tab and see the raw yt-dlp command line output in the Output tab. If you only want certain videos, instead choose Check channel. This will grab all the metadata for the channel's videos, displaying them as a grid of thumbnails, and then you can select them through the GUI and download the specific ones you want. It also might be a good idea to do this if you want to test the options on one video to make sure you're getting the result you want before going all-in on downloading the channel.

Looking over the yt-dlp output as a sanity-check, I can confirm it does the following things:

1. Writes en.vtt subtitles (English subtitles in the default YouTube format)

2. Selects the best video format (1080p VP9)

3. Selects the audio format 251-1 (which is the best Japanese audio on this particular video)

4. Converts subtitles to SRT

5. Merges all three tracks into MKV

6. Deletes external SRT

which I think is all the functionality you requested! Let me know if you have any further questions and I'll do my best to answer them.