Objectify: Overview
This is part of a series of articles from J. Scott Edwards about his pet project Objectify.
– Editor
History
Right after Y2K I decided to resurrect a project I had started in the late ’70s to write an operating system. I started a project called EasyOS (1) on SourceForge to develop an OS based upon the best parts of Linux, FreeBSD, OpenBSD, etc. After toying around for a while I decided that part of the problem was that current operating systems are basically the same as the operating systems I had worked with in the 1970s, with the addition of a GUI. One example would be that all data is stored in files.
I decided that instead of using the same paradigms as existing operating systems, I needed to rethink those paradigms. I started another project: New World OS on SourceForge and in December 2004, I wrote an article for OS News about some of my ideas.
When it came time to start developing the system, I debated whether I should start completely from scratch; develop it from the hardware interface level and work my way up, or if I should start by testing some of my ideas on an existing system. If I started on top of an existing system I could see if my ideas were feasible, before dumping years worth of time into them only to discover that I had overlooked some fundamental flaw. I decided it made the most sense to test out some ideas on top of another OS first. To do releases on SourceForge I needed a package name. New World OS didn’t seem like a good package name because it wasn’t really an OS yet. I decided to call the package Objectify because it stored everything as an object. I often think that I should have called it Insomnia because almost all of it was developed while I should have been sleeping.
I started out storing the objects in files, each object in a separate file, but that proved to be cumbersome. I then tried storing all of the objects in a “sparse” file. Oddly, that ended up consuming gobs of disk space. Since one of my main ideas was to store objects directly on the hard drive and NOT in files, I ultimately decided to “just do it” and started reading and writing objects directly to an entire hard drive (like /dev/hdb) or a disk partition (like /dev/hda7). This worked quite well on Linux, but hasn’t worked on other operating systems like FreeBSD and OpenBSD, so I added the capability to substitute a regular file (like ~/.private.obj) for the block device (2).
One of the fundamental ideas, if not THE fundamental idea, was to eliminate storing data in files. I wanted to store data in a format more natural to the computer, so that a program did not have to read and convert it’s data from a file, do its thing, and then convert the data back to a file. I wanted the program to store it’s data in objects and then, only if the data had to be exchanged with another device, convert it to a file.
While I was working on the project, I had a collection of documents that I was scanning into the computer so I could shred the paper copies. However, since my project was not going to be ready to store these documents for some time, and I did not want to have the files sitting around on my computer or on DVD-R without encryption, I realized I could create a generic “FILE” object that would just store the file data until it was ready. I began storing my important documents in this manner in 2006 and now have over 3,500 files stored.
In January of 2007 I decided it would be nicer if the Objectify package had it’s own project on SourceForge, so I created an Objectify project. However, I have not completed the move so the CVS repository and the released packages are still under the New World OS project. The only thing I have in the Objectify project so far are the Bug and Feature Request Trackers.
I find it ironic that my initial goal was to eliminate the storing of data in files but currently that is the most developed function of the project. However, it is interesting to note that Objectify is inverted from other systems; instead of storing objects in files, it stores files in objects.
Notes:
- In early 2005 someone else wanted to use the EasyOS moniker on SourceForge. Since I was concentrating on New World OS by that time, I let them take it over. So the EasyOS project that is now on SourceForge has no relation to my project.
- I have tested the code on FreeBSD, OpenBSD, and Cygwin. I have started trying to get it to compile on Mac OS X but there were a couple of POSIX functions that are missing on OS X and I haven’t had time to work through it.
Loading ...
What advantage is this change supposed to have?
Hi Dave,
Thank you for asking. Some of the advantages I am trying to achieve are:
1) Simplified and more efficient program code. Because every single program that reads a file has to have code to convert the data in a file into the format the program uses (I am going to assume that the program is object oriented so it has to convert the data to objects). Then when the program is finished it has to convert the objects back into the file format. It seems to me that it would be simpler to store the data in the format native to the program so it can read and write the objects directly without having to do all the fooling around with converting. If the data needs to be converted to a file, have a separate program that knows how to do that. That seems like it would be more efficient because then if you have 50 programs that have to manipulate the data, none of them have to know about the file format or how to convert it. You only need one program that has to know how to convert the data to a file.
I admit that you can simplify the file conversion somewhat with libraries, but there always has to be some conversion.
2) Eliminate (or reduce the number of) file names. Maybe I am the only one that thinks this, but it seems like the whole file name thing is a pain in the a**. I get really tired of having to make up a file name every time I store a piece of data. And then later when I am trying to find that data, having to remember what I called it and where I stored it.
I would much rather just be able to say “list all of the music I have by The Beatles”, “what photos do I have of Frank”, “show me last month’s Rocky Mountain Power statement”. Instead of trying to remember if I named it rckymtnpwr-080731.pdf, rocky_mtn_pwr-2008_07_31.pdf, or rocky_mountain_power-July_31_2008.pdf, what directory it is in, and on which computer.
3) Where did this file come from? Again, maybe it is just me, but often I have some file and I want to know where it came from. If I download some code from somewhere on the Internet, I may need some other file to go with it and I can’t remember exactly what URL I downloaded it from. Then I have to waste time locating the source again so I can what I need. It would be easier if I could just ask “where did this file come from?”.
Another example of this idea, is that I want to wire Objectify into the GIMP so that when I make changes to a photo for example it associates the two photos together. Later if I make other changes to either of those photos it associates it as well. I also hope to keep track of the changes made to the various photos. If I make a new photo out of three other photos it keeps track of those photos so I know how the new photo was created and get back to its origins.
4) Everything in one place. I would like to have all of my information in one place. I don’t want to have to search all over the place to find something. If it is all in one place, all I have to do is look in that one place and I would like it if it was as easy as possible to find in that one place.
——————————
I know what I have now is a long, long way from what I hope to achieve. And it may turn out in the end that everything I have imagined may not work. But I have found it helpful to me even in it’s current state.
Let me give an example: in previous years when I went to the tax accountant I he would inevitably ask for information that I did not have with me, usually many things. I would make a list, go home and dig through boxes trying to find it the information, and then we would play phone tag for a while and a week or two later he could finish my taxes. This year I took my laptop with Objectify and I was able to answer every single question he asked on the spot. To me that alone was worth it. (Unfortunately, I still had to deal with file names to do it, but maybe someday…)
Definitely interesting. I’ll definitely follow this… hope it one day gets to a usable OS state.