实现一个Storage Manager，用于管理文件中的数据对象。
Requirement
In this programming assignment, you need to implement a storage manager that maintains a series of data objects in each data file. You first need to install and run Eclipse on your machine and import the “storage manager” project (see Appendix A). Please generate an API document (see Appendix B) and then take a look at that document as well as the source code to familiarize yourself with this assignment. This assignment provides you with a set of incomplete classes (in particular, see SlottedPage, FileManager, and BufferedFileManager which contain methods currently throwing an UnsupportedOperationException). You will need to write code for these classes. Your code will be graded by running a set of unit tests and then examining your code (see SlottedPageTest and FileManagerTest which use JUnit, as well as BufferedFileManagerTest which produces output messages). For details of running these tests, refer to Appendix A. Note that passing unit tests does NOT necessarily guarantee that your implementation is correct and efficient. Please make sure that your code is correct and efficient so that it will not cause any problem in many other cases not covered by the unit tests. If you have questions, please contact the TA(s) or the instructor. The remainder of this document describes the components that you need to implement.
Part 1. The Slotted Page Structure
The slotted page structure allows a storage manager to store variable-length records (data objects) within a fixed size page (an in-memory copy of a disk block). Each page is essentially a byte array that stores a header (at the beginning) and a number of data objects (at the end). The header of a page consists of (1) a 4-byte integer representing the number of entries in the page for storing data objects, and (2) a series of 4-byte integers (one for each data object) each of which represents where the corresponding data object is stored within the page (i.e., the location of the corresponding data object within the page). Each page has, between its header and data objects, a free space where more data objects and header entries can be added. For further details of the slotted page structure, refer to Section 10.5.2 of the textbook. Upon a request to delete a data object from a page, the corresponding header entry (for storing the location of the data object) is simply set to -1 to indicate that there is no associated data object (the actual object is NOT necessarily removed from the page).
In this part, you need to implement the following methods (your code needs to pass all of the 5 tests in SlottedPageTest):
add(Object o): adds the specified object o in the SlottedPage on which this method is invoked. This method must first save the object o in the free space of the SlottedPage by calling the save(Object o) method. This save(Object o) method returns an int value indicating the location at which the object o is saved in the SlottedPage. That int value must be stored at the right header entry of the SlottedPage so that the saved object o can be accessed in the future. For example, assume that object “123” is saved at location 2038 in the byte array of an empty SlottedPage. Then, the first 4 bytes of the byte array (i.e., the beginning of the header) must store an int value 1 to indicate that there is only one entry in the header (in response to the addition of object “123”). The next 4 bytes of the byte array (i.e., the 0th entry in the header) must store 2038 (i.e., the location at which “123” is saved). When an additional object “456” is saved at location 2028 in the byte array, the first 4 bytes of the header must store 2 (to indicate that there are two entries in the header) and the 1st entry (i.e., the entry after the 0th entry) in the header must store 2028 (i.e., the location at which “456” is saved). To find the number of entries that the SlottedPage currently has, use the entryCount() method. To set the number of entries in the header to an int value, use the setEntryCount(int count) method. To save location l at the ith entry in the header, call saveLocation(i, l).
get(int index): returns the object at the specified index in the SlottedPage on which this method is invoked. For example, get(0) returns the object at index 0 (i.e., the object whose location is stored at the 0th entry in the header) and get(1) returns the object at index 1. This method must first find the location of the object at the specified index by calling the getLocation(int index) method. This getLocation(int index) method returns the int value stored at the header entry specified by index (i.e., the index-th header entry). If that location is -1, meaning that the object was removed from the SlottedPage, the method needs to return null. Otherwise, it needs to obtain the object by calling the toObject(byte[] b, int offset) method and then return that object. If the given index cannot match any of the entries in the SlottedPage (e.g., get(-1)), the method needs to throw an IndexOutOfBoundsException.
remove(int index): removes the object at the specified index from the SlottedPage on which this method is invoked. This method must save -1 at the appropriate entry in the header. This method must also return the object removed (i.e., the object previously stored at the specified index).
iterator(): returns an iterator over the objects (excluding those removed) stored in the SlottedPage on which this method is invoked. To find the number of entries in the current SlottedPage, use entryCount(). To get the object at each index, call get(int index). Note that get(int index) returns null if there is currently no object at the specified index due to the deletion of the previous object. Feel free to add an auxiliary class or data structure for this iterator() method.
compact(): reorganizes the SlottedPage on which the method is invoked (in order to maximize the free space of that SlottedPage). This method is used by the save(Object o) method when the object to save cannot fit into the current free space of the SlottedPage. This method needs to move objects at the end of the SlottedPage in a manner that eliminates the space previously wasted by the objects removed from the SlottedPage.
Please make sure that your code passes all of the tests in SlottedPageTest. 2
Part 2. The Basic Storage Manager Implementation
In this part, you need to implement, in FileManager.java, a basic storage manager that maintains a series of data objects in each data file without buffering (i.e., reading/writing SlottedPages directly from/to data files). The methods to complete are as follows (your code needs to pass all of the 5 tests in FileManagerTest):
fileID(String fileName): returns an integer-type ID of the file specified by fileName. These IDs are consecutive integers that start from 0. For example, a FileManager may assign ID 0 to a file named “table0.dat” and ID 1 to a file named “table1.dat”. In this case, to access data objects stored in “table0.dat”, integer-type ID 0 needs to be used. Similarly, integer-type ID 1 must be used to access data objects stored in table1.dat. Such relationships from file names to integer-type IDs (e.g., “table0.dat” to 0 and “table1.dat” to 1) are stored in a map named fileName2ID. If the argument fileName of the method (i.e., fileID(String fileName)) is already registered in fileName2ID, the method needs to return the ID associated with fileName in fileName2ID (e.g., return 0 when fileName is “table0.dat”). Otherwise, the method needs to assign a new ID to the file specified by fileName. Furthermore, it needs to create a SlottedPageFile for accessing the data file specified by fileName and then register that SlottedPageFile in the id2file map (which associates each file ID with the corresponding SlottedPageFile). For details of SlottedPageFile, refer to SlottedPageFile.java.
put(int fileID, Long location, Object o): puts object o at location location in the file specified by fileID. Here, location has a long-type value, whose first half (4 bytes corresponding to an integer) represents the ID of the page (e.g., page 0, page 1, etc.) and the second half represents the index within the page. For example, put(10, 0x00000001L, o) stores object o in page 0 of the file specified by ID 10 (i.e., the first disk block in the file) at index 1 within the page. On the other hand, put(10, 0x00010000L, o) stores object o in page 1 of the file specified by ID 10 (i.e., the second disk block in the file) at index 0 within the page. Given a long-type argument location, use first(location) to get the ID of the page and second(location) to get the index within the page. After finding an appropriate page p, call put(int index, Object o) on p to put the object in that page and then call the updated(p, fileID) method of FileManager to indicate that the page p is updated (then the updated(p, fileID) method of FileManager automatically writes the page to the appropriate data file). If the location argument has an appropriate value (e.g., its first half refers to page -1), then put(int fileID, Long location, Object o) needs to throw an InvalidLocationException.
get(int fileID, Long location): returns the object at location location in the file specified by fileID.
remove(int fileID, Long location): removes the object at location location from the file specified by fileID.
iterator(int fileID): returns an iterator over all objects stored in the the file specified by fileID. This method needs to use page(int fileID, int pageID) of FileManager and iterator() of SlottedPage. Make sure this method efficiently uses the memory (e.g., does NOT put all of the objects in the memory first thereby incurring high space overhead and then return an iterator over these objects).
Please verify your code using the tests in FileManagerTest.
Part 3. Buffer Management
The FileManager class implemented in Part 2 directly accesses data files (i.e., no buffering), causing disk seeks frequently. In this part, you need to implement the BufferedFileManager class so that it extends the functionalities of FileManager in a manner that benefits from buffering (i.e., frequently used pages are kept in memory, thereby enabling fast read and write operations). Furthermore, you need to implement a page eviction policy for the cases where there are too many pages to keep in the main memory. The choice of eviction policy is up to you. It is not necessary to do something sophisticated. Describe your policy in the document that you submit.
When BufferedFileManager is implemented correctly, BufferedFileManagerTest will produce some output as follows (the exact number of reads and writes may vary depending on the buffering strategy; however, there should in general be less reads and writes as the buffer size increases):
buffer size: 4 pages
1000 additions % [{name: table0.dat, reads: 0, writes: 38}]
10 removals % [{name: table0.dat, reads: 10, writes: 48}]
iteration over 990 elements % [{name: table0.dat, reads: 52, writes: 52}]
shut down % [{name: table0.dat, reads: 52, writes: 52}]
buffer size: 16 pages
1000 additions % [{name: table0.dat, reads: 0, writes: 26}]
10 removals % [{name: table0.dat, reads: 9, writes: 35}]
iteration over 990 elements % [{name: table0.dat, reads: 48, writes: 51}]
shut down % [{name: table0.dat, reads: 48, writes: 51}]
buffer size: 64 pages
1000 additions % [{name: table0.dat, reads: 0, writes: 0}]
10 removals % [{name: table0.dat, reads: 0, writes: 0}]
iteration over 990 elements % [{name:table0.dat, reads: 0, writes: 0}]
shut down % [{name: table0.dat, reads: 0, writes: 42}]
Appendix A. Installing Eclipse and Importing a Java Project
- Visit:
http://www.eclipse.org/downloads/ - From the web site, download the eclipse installer (those for Linux, Windows, and Mac OS X are available) and then choose “Eclipse IDE for Java Developers” and install it.
- After finishing installation, start Eclipse.
- When Eclipse runs for the first time, it asks the user to choose the workspace location. You may use the default location.
- In the menu bar, choose “File” and then “Import”. Next, select “General” and “Existing Projects into Workspace”. Then, click the “Browse” button and select the “storage manager.zip” file contained in this assignment package.
- Once the project is imported, you can choose one among SlottedPageTest.java, FileManagerTest.java, and BufferedFileManagerTest.java in the storage.test package and then run it by clicking the icon highlighted in