Really? Idle, my Jellyfin instance has about 260MB resident right now. I don't think I've had any memory issues while in use; the old Mac Mini (running Linux) I have it running on only has 4GB total, and it's also running a couple Mono-based apps that are a bit more memory-hungry.
Jellyfin runs on about 700-800MiB of RAM for me. It sounds like you're hitting a memory bug if you're running out of RAM just serving files, to be honest. I've never used Plex but I doubt it runs with just 70MiB.
Perhaps it'll use much more RAM transcoding, but I'd still expect a dedicated 2GiB to be plenty for Jellyfin even in extreme cases. I doubt you'll get any decent transcoding speed out of ffmpeg if you need a buffer that size anyway.
