ya got me i had a windows 98 pc in my adolescence and was basically forced to learn how a computer works, and maybe thats not for everyone, but when people dont know what a filesystem is that makes me personally upset and thats whats important
Why do people obsess over file systems? They're not actually very useful. There is no law of nature that says "An operating system must have a built-in file system!" The Windows and Linux file systems are garbage. They're simultaneously used for the operating system itself, program data, and documents and pictures. That's nonsensical. Data should be stored in a data structure that makes sense. You can barely even do file search on Windows and Linux, but every phone app has pretty sophisticated features for instantly searching file metadata in pictures and so on. To even have a photo library on any system you have to build an SQL database of the photos, otherwise it's far too slow to search through them. File systems are really not that useful. And they only complicate the kernel and operating system features. The only reason Linux has such a prominent file system is because that was all the hotness back then with turning everything into a file (which turns out is not actually that useful compared to just having useful APIs like iOS and Android have).
Because they might as well not even be file systems. They're just data structures that the OS developers control for storing data. You can run Linux with an almost empty tmpfs root file system, and barely touch the file system at all. There's nothing fundamental about file systems other than being the most prominent way to allocate and track persistent storage space.
It sounds like SQLite still requires a couple dynamically sizeable files for storing the database, so it probably isn't possible. You'd have to have a key-value store that can allocate blocks for the different files it needs.
It seems really bizarre to me, because you'd think a large tech company would have found a performance reason for throwing out the file system and disk partitions and using the device directly for storage, but it doesn't seem like that's happened yet. Probably wouldn't even be that hard for a team at a big company to implement a basic file system stripped of all the hierarchical/metadata stuff and use that.
I guess it really comes down to the fact that every application needs something like the program heap to store variable sized objects, and the file system is the closest thing to that currently.
Key-value stores, relational databases, logs, anything you can think of. It just depends on what you're storing. There just aren't a lot of data structures implemented because file systems work well enough. I think it would be cool if there was a block storage alternative to file systems that was more like virtual memory.
For example: Pangolin: A Fault-Tolerant Persistent Memory Programming Library: https://www.usenix.org/conference/atc19/presentation/zhang-lu
Only designed for embedded devices though.
File systems are especially annoying because they have all sorts of metadata that makes them less reliable. Does a file have the correct permissions? Is it writable? Is another program using it? Is it fragmented?
An iOS or Android app has no use for file permissions or ownership or creation date or even sub-directories, in its own sandbox directory, so that's just extra overhead and extra code (in the file system driver) that provides no benefit, but can still potentially cause problems.
I don't know if I've ever seen a single Windows application that correctly handles every error case when using the file system. You'll get all sorts of unfriendly errors, crashes, and freezes because of stupid things like file permissions or some obscure file system "feature". And that doesn't even get to file path string encoding, maximum files in a directory, max file size, etc. that also have to be considered when writing robust software. And every thing that isn't considered is another thing that will end up hurting the end user. And all of these things vary slightly between operating systems, and developers don't typically use a custom file system on top of the OS file system, so it means there isn't a consistent programming environment, and even more error cases have to be considered (and these types of things do produce real errors that affect users). How often do SQL databases and key-value stores produce errors? (probably only as often as the file system they're implemented on top of do)
ya got me i had a windows 98 pc in my adolescence and was basically forced to learn how a computer works, and maybe thats not for everyone, but when people dont know what a filesystem is that makes me personally upset and thats whats important
Why do people obsess over file systems? They're not actually very useful. There is no law of nature that says "An operating system must have a built-in file system!" The Windows and Linux file systems are garbage. They're simultaneously used for the operating system itself, program data, and documents and pictures. That's nonsensical. Data should be stored in a data structure that makes sense. You can barely even do file search on Windows and Linux, but every phone app has pretty sophisticated features for instantly searching file metadata in pictures and so on. To even have a photo library on any system you have to build an SQL database of the photos, otherwise it's far too slow to search through them. File systems are really not that useful. And they only complicate the kernel and operating system features. The only reason Linux has such a prominent file system is because that was all the hotness back then with turning everything into a file (which turns out is not actually that useful compared to just having useful APIs like iOS and Android have).
Android and iOS still have file systems, the user just isn't allowed to look at them
even on mac they make it obnoxiously hard to browse outside of a short list of curated user folders.
Because they might as well not even be file systems. They're just data structures that the OS developers control for storing data. You can run Linux with an almost empty tmpfs root file system, and barely touch the file system at all. There's nothing fundamental about file systems other than being the most prominent way to allocate and track persistent storage space.
Has anybody tried making a partition that’s really just a sqlite3 binary blob?
It sounds like SQLite still requires a couple dynamically sizeable files for storing the database, so it probably isn't possible. You'd have to have a key-value store that can allocate blocks for the different files it needs.
It seems really bizarre to me, because you'd think a large tech company would have found a performance reason for throwing out the file system and disk partitions and using the device directly for storage, but it doesn't seem like that's happened yet. Probably wouldn't even be that hard for a team at a big company to implement a basic file system stripped of all the hierarchical/metadata stuff and use that.
I guess it really comes down to the fact that every application needs something like the program heap to store variable sized objects, and the file system is the closest thing to that currently.
and what is the alternative to having a file system?
Key-value stores, relational databases, logs, anything you can think of. It just depends on what you're storing. There just aren't a lot of data structures implemented because file systems work well enough. I think it would be cool if there was a block storage alternative to file systems that was more like virtual memory.
For example: Pangolin: A Fault-Tolerant Persistent Memory Programming Library: https://www.usenix.org/conference/atc19/presentation/zhang-lu
Only designed for embedded devices though.
File systems are especially annoying because they have all sorts of metadata that makes them less reliable. Does a file have the correct permissions? Is it writable? Is another program using it? Is it fragmented?
An iOS or Android app has no use for file permissions or ownership or creation date or even sub-directories, in its own sandbox directory, so that's just extra overhead and extra code (in the file system driver) that provides no benefit, but can still potentially cause problems.
I don't know if I've ever seen a single Windows application that correctly handles every error case when using the file system. You'll get all sorts of unfriendly errors, crashes, and freezes because of stupid things like file permissions or some obscure file system "feature". And that doesn't even get to file path string encoding, maximum files in a directory, max file size, etc. that also have to be considered when writing robust software. And every thing that isn't considered is another thing that will end up hurting the end user. And all of these things vary slightly between operating systems, and developers don't typically use a custom file system on top of the OS file system, so it means there isn't a consistent programming environment, and even more error cases have to be considered (and these types of things do produce real errors that affect users). How often do SQL databases and key-value stores produce errors? (probably only as often as the file system they're implemented on top of do)