A multifile is a file that contains a set of files, similar to a .zip or .rar archive file. They are meant for containing multiple resources such as models, textures, sounds, shaders, and so on, and Panda can load them directly from the multifiles without having to unpack them first. Many games employ a similar concept of “data” file such as .upk for Unreal Engine and .pak for Quake Engine.
The multify program¶
The multify console program creates such files. You can get information about
the commandline parameters by running multify with the
-h option. This is
how program describes itself:
Usage: multify -[c|r|u|t|x] -f <multifile_name> [options] <subfile_name> ...
multify is used to store and extract files from a Panda Multifile. This is similar to a tar or zip file in that it is an archive file that contains a number of subfiles that may later be extracted.
Panda’s VirtualFileSystem is capable of mounting Multifiles for direct access to the subfiles contained within without having to extract them out to independent files first.
The command-line options for multify are designed to be similar to those for tar, the traditional Unix archiver utility.
If you want to prepare to read assets from a Multifile directly, you can “mount” it into the virtual file system:
VirtualFileSystem *vfs = VirtualFileSystem::get_global_ptr(); vfs->mount("./foo.mf", ".", VirtualFileSystem::MF_read_only);
If you want to read assets, you can mount a whole directory structure from a webserver.
If your webserver hosts:
Put this in your config.prc:
vfs-mount-url http://myserver/mydir /mydir model-path /mydir
Or, equivalently, write this Python code at startup:
vfs.mount(VirtualFileMountHTTP('http://myserver/mydir'), '/mydir', 0) getModelPath().appendDirectory('/mydir')
and then you can load models like this in your Python code:
model = loader.loadModel('models/myfile.bam') texture = loader.loadTexture('maps/mytexture.png')
If you want to prepare for reading and writing assets to a Multifile do the following.
from panda3d.core import VirtualFileSystem from panda3d.core import Multifile from panda3d.core import Filename mf = Multifile() mf.openReadWrite("models.mf") vfs = VirtualFileSystem.getGlobalPtr() if vfs.mount(mf, ".", VirtualFileSystem.MFReadOnly): print('mounted')
If you want to prepare for reading and writing assets to a ‘subdirectory’ Multifile do the following. Note “mysys” must always be literally written in any python code. E.g. “mysys/memfdir/mfbar2.txt”
from panda3d.core import VirtualFileSystem from panda3d.core import Multifile from panda3d.core import Filename mf = Multifile() mf.openReadWrite("models.mf") vfs = VirtualFileSystem.getGlobalPtr() if vfs.mount(mf, "mysys", VirtualFileSystem.MFReadOnly): print('mounted')
If you are having problems loading from multifiles you can list the complete contents of your .mf file with a command like:
multify -tvf mymultifile.mf
Doing a sanity inspection like this can be useful to ensure that your assets are in the right place within the multifile.
Multifile class is designed for opening, reading and
writing multifiles. You can open a new multifile by creating an instance of the
class and calling the
from panda3d.core import Multifile mf = Multifile() mf.openRead("foo.mf")
openRead method opens the multifile as read-only. If you want to make
changes to it and write it back to disk, you will need to use the
openReadWrite method. Also, there exists
openWrite to create a new
If you have made important structural changes to a Multifile, it is recommended
to rewrite the multifile using the
repack() method. (This won’t work if
you’ve opened it using
openRead.) If you are uncertain about whether it has
become suboptimal, you can call
needsRepack() which returns True if the
Multifile is suboptimal and should be repacked.
To write it back to disk, you can use the
flush() method which flushes the
changes you’ve made to the multifile back to disk, or the
close() method if
you’re done with the file.
To mount Multifile objects into the VirtualFileSystem without writing them to disk first, here’s an example on how to mount them:
yourMF = Multifile() #... now do something with yourMF vfs = VirtualFileSystem.getGlobalPtr() vfs.mount(yourMF, ".", VirtualFileSystem.MFReadOnly)
Files that are added to a multifile are called subfiles. You can add existing
files to a multifile object using the
addSubfile function. This function
takes three arguments: the target filename, the existing source file and the
compression level (1-9). There is also
updateSubfile, which does the same
thing but if the file already exists, only updates it if the content is
There are several other methods which operate on subfiles, which you can find on
Multifile page in the API Reference.
Here are a few examples of working with subfiles:
from panda3d.core import VirtualFileSystem from panda3d.core import Multifile from panda3d.core import Filename m = Multifile() # Add an existing real os file with compression level 6 m.openReadWrite("foo.mf") m.addSubfile("bar.txt", Filename("/tmp/bar.txt"), 6) m.flush() # Destroy the contents of the multifile # Add an existing real os file to be the first multifile m.openWrite("foo.mf") m.addSubfile("bar.txt", Filename("/tmp/bar.txt"), 6) m.flush() # Permanently re-order in ascending order the # directories and files in the multifile m.openReadWrite("foo.mf") m.repack() m.flush() # Open a multifile and replace the contents of the mulifile file # with new contents m = Multifile() m.openReadWrite("foo.mf") m.updateSubfile("bar.txt", Filename("/tmp/bar2.txt"), 9) m.flush() # Open a multifile and extract all files smaller than 3kb # New real os files are created with the contents of the multifile data m = Multifile() m.openRead("foo.mf") for i in range(m.getNumSubfiles()): if m.getSubfileLength(i) < 3 * 1024: m.extractSubfile(i, Filename("/tmp/" + m.getSubfileName(i))) # Find, print and remove a file named bar.txt barIdx = m.findSubfile("bar.txt") if barIdx != -1: # It returns -1 if it doesn't exist print(m.readSubfile(barIdx)) m.removeSubfile(barIdx) m.flush() m.close()
std::ostringstream os (std::ios::in | std::ios::out); std::istream is (os.rdbuf ()); os.write((char*)&stuff, sizeof(stuff)); PT(Multifile) mf = new Multifile(); mf->open_write(fileName); mf->add_subfile("foo.mf", &is,6); mf->flush(); mf->close();
If the foo.mf file were to have a contained bar.egg.pz file, load the egg and use it similar to other model loading methods.
nodepath = loader.loadModel("foo/bar")
Multifile algorithms are stream-based and not random-based. In a running game,
from the output, if a message is received saying something similar to the words
seek error for offset then a file in the multifile is trying to be accessed
by a random-based method. For multifiles and fonts, an example of a random-based
file is an .rgb file. An alternative different from the use of an .rgb file is
the use of a .ttf instead. An example follows.
# models is the original directory # models.mf it the new target multifile multify -c -f models.mf -v models
In the game, from the multifile models.mf, load the .ttf file.
font = loader.loadFont("models/arial.ttf")
Multifiles can also encrypt your files with a password. To do so, you need to
set the encryption flag and password using the
setEncryptionPassword methods, before adding, extracting or reading
At the OS prompt, to create a password protected multifile and print out the contents do the following.
# models is the original directory # models.mf it the new target multifile multify -c -f models.mf -ep "mypass" -v models
This code creates a multifile and adds an encrypted file to it:
m = Multifile() m.openReadWrite("foo.mf") m.setEncryptionFlag(True) m.setEncryptionPassword("foobar") # Add a new file to the multifile m.addSubfile("bar.txt", Filename("/tmp/bar.txt"), 1) m.flush() m.close()
You can read encrypted multifiles the same way:
m = Multifile() m.openRead("foo.mf") m.setEncryptionFlag(True) m.setEncryptionPassword("foobar") # Prints the contents of the multifile print(m.readSubfile("bar.txt"))
At the OS prompt, to see the contents of a password protected multifile perform
multify -tvf models.mf -p "mypass"
You can test the reading in a of password-protected multifile, followed by the mounting of the file using the following code.
from panda3d.core import Multifile mf = Multifile() mf.openRead("models.mf") mf.setEncryptionFlag(True) mf.setEncryptionPassword("mypass") from panda3d.core import VirtualFileSystem vfs = VirtualFileSystem.getGlobalPtr() if vfs.mount(mf, ".", VirtualFileSystem.MFReadOnly): print('mounted')
When running the game, the following should be seen:
You can check if a certain subfile is encrypted or not using the
isSubfileEncrypted method, which takes the subfile index as parameter.
It is possible to have a multifile where different subfiles have different
encryption, but you will not be able to mount it with the VirtualFileSystem or
use it with the multify tool. To mount an encrypted file using the
VirtualFileSystem, pass the password as parameter to the
VirtualFileSystem *vfs = VirtualFileSystem::get_global_ptr() vfs->mount("./foo.mf", ".", VirtualFileSystem::MF_read_only, "foobar");
To use encryption with the multify tool, run it with the
-e option, which
will prompt for a password on the command line. Alternatively, if you also
-p "password" option, you can specify it in the command instead
of typing it at the prompt.