Extract and process resources from Quake-style pak files.
The extract_resources() and process_resources() functions of this module enable programmatic extraction and (optional) processing of resources from one or more pak files.
The resource_names() function retrieves a set of all the resource names in one or more pak files.
All of these functions have a sources parameter which can accept either a string specifying the filepath of a single pak file to process, or an iterable container of strings specifying multiple pak files to process.
Resource selection (using a set of names or a name map) and processing (with a user-provided function hook) is described in more detail in the documentation for each function.
The return status of extract_resources()/process_resources() is an indicator of whether exceptions were encountered while reading the pak file or processing resources. For the simple “extract everything” uses, this return status maps directly to whether all of the intended resources were extracted. For usage patterns that explicitly supply an input set or dict of resources however, you should check that container after the invocation to see which resources were not handled (if any).
Example of retrieving resource names from a pak file:
pak0_resource_set = expak.resource_names("pak0.pak")
Example of extracting all resources from multiple pak files:
expak.extract_resources(["pak0.pak", "pak1.pak"])
Example of extracting specified resources from multiple pak files:
sources = ["pak0.pak", "pak1.pak"]
targets = set(["sound/misc/basekey.wav", "sound/misc/medkey.wav"])
expak.extract_resources(sources, targets)
More complex example:
# Extract sound/misc/basekey.wav, convert it to OGG format, and save the
# result as base_key.ogg. Similarly process sound/misc/medkey.wav to
# create medieval_key.ogg. Look for those resources in pak0.pak and in
# pak1.pak.
def ogg_converter(orig_data, name):
new_data = my_ogg_conversion_func_not_shown_here(orig_data)
with open(name, 'wb') as outstream:
outstream.write(new_data)
return True
sources = ["pak0.pak", "pak1.pak"]
targets = {"sound/misc/basekey.wav" : "base_key.ogg",
"sound/misc/medkey.wav" : "medieval_key.ogg"}
expak.process_resources(sources, ogg_converter, targets)
# Notify if some of the desired files were not created.
if targets:
print("not found (or not successfully processed):")
for t in targets:
print(" " + t)
Finally, here’s a complete script that can be used to create copies of bsp files and modify them according to the entity descriptions contained in a set of .ent and/or .map files. The most common example of this usecase is modifying maps for a CTF server so that they include flags and CTF-specific spawnpoints (using the entity files provided by the Threewave CTF server package).
This procedure also requires the qbsp utility.
import sys
import glob
import subprocess
import expak
# Name of qbsp utility to use, and complete path if required.
QBSP = "qbsp"
# Extensions used to find entity files. The code below requires all of
# these extensions to be the same length.
ENT_EXTS = [".ent", ".ENT", ".map", ".MAP"]
ENT_EXT_LEN = len(ENT_EXTS[0])
# Prefix to use when looking for bsp files in the paks. Probably shouldn't
# change this!
MAPS_PRE = "maps/"
MAPS_PRE_LEN = len(MAPS_PRE)
# Docstring processors can mistreat backslash-n in example code blocks.
LF = chr(10)
def usage(script_path):
print("")
print("Extract bsp files from pak files and apply entity changes.")
print("")
print("Entity files (.ent or .map) will be discovered in the working ")
print("directory when you run this script. An entity file intended ")
print("to modify foo.bsp should be named foo.ent or foo.map.")
print("")
print("Specify paths to pak files (containing the bsp files) on the ")
print("command line:")
print(" {0} <pak_a.pak> [<pak_b.pak> ...]".format(script_path))
print("")
def main(paks):
# Get entity files from the working directory. Search for all valid
# extensions, and (to accomodate case-insensitive platforms) form a
# set from the aggregate results to make sure we don't have duplicates.
# This isn't super-efficient but it's straightforward.
ents = set()
for ext in ENT_EXTS:
ents.update(glob.glob("*" + ext))
# Create a map of bsp resource names to entity files. Ensure that there
# is a one-to-one relationship.
ents_for_bsps = dict([(MAPS_PRE + e[:-ENT_EXT_LEN].lower() + ".bsp", e)
for e in ents])
if len(ents) != len(ents_for_bsps):
sys.stderr.write("error: multiple entity files in the working "
"directory would apply to the same bsp" + LF)
return 1
# Form a set of the resources we want to process. This set will be
# modified to indicate which ones are left unprocessed.
bsps = set(ents_for_bsps.keys())
# Define our converter. qbsp requires a file for input, so we'll write
# out the file and then invoke qbsp.
def converter(orig_data, name):
print("extracting " + name)
# Dump the file in the working directory, not in a maps subfolder.
expak.nop_converter(orig_data, name[MAPS_PRE_LEN:])
# Run qbsp. Capture the output (which is all to stdout, even in an
# error case).
print("applying " + ents_for_bsps[name])
p = subprocess.Popen([QBSP, "-onlyents", ents_for_bsps[name]],
stderr=subprocess.STDOUT,
stdout=subprocess.PIPE)
qbsp_out = p.communicate()[0]
qbsp_result = p.returncode
# Overall success status = qbsp success status.
if qbsp_result == 0:
return True
# If there was a problem, dump the qbsp output.
sys.stderr.write("error: qbsp reported a problem:" + LF)
sys.stderr.write(qbsp_out)
return False
# Now that we have our pak sources, converter func, and target
# resources... do the processing.
expak.process_resources(paks, converter, bsps)
# Inform if there were bsps that we didn't find.
print("")
if bsps:
print("not found (or not successfully processed):")
for b in bsps:
print(" " + b)
else:
print("all bsps successfully found and processed")
print("")
if __name__ == "__main__":
script_path = sys.argv[0]
paks = sys.argv[1:]
if not paks:
usage(script_path)
sys.exit(1)
sys.exit(main(paks))
Extract and process resources contained in one or more pak files.
The converter parameter accepts a function that will be used to process each selected resource that is found in a pak file. This converter function accepts the resource binary content and a name string as arguments. It returns a boolean success status, which indicates whether the resource was processed but does not affect the overall return value of process_resources. The converter function may also raise any exception to stop processing a given resource; this does not immediately interrupt the overall process_resources loop, but will cause process_resources to return False when it finishes.
The nop_converter() function in this module is an example of a simple converter function that just writes out the resource content in its original form.
The selected resources, and the name passed to the converter function for each, depend on the type and content of the targets argument:
If the targets argument is a set or dict, the element corresponding to each found and successfully processed resource is removed from it.
This function will return True if each specified source is a pak file, is read without I/O errors, and is processed without converter exceptions. False otherwise.
Note
In the case where targets is not None, a True result does not indicate that all of the resources in targets were found and processed. And if False is returned, some processing may have been done. Examining the contents of targets after the function returns is a good idea.
| Parameters: |
|
|---|---|
| Returns: | True if no IOError exception reading the pak file and no exception processing any resource, False otherwise |
| Return type: | bool |
Extract resources contained in one or more pak files.
Convenience function for invoking process_resources() with the nop_converter() function as the converter argument.
See process_resources() for more discussion of the return value and the handling of the targets argument.
| Parameters: |
|
|---|---|
| Returns: | True if no IOError exception reading the pak file and no exception extracting any resource, False otherwise |
| Return type: | bool |
Return the name of every resource in one or more pak files.
Return a set of resource name strings collected from all of the given pak files, if each specified file is a pak file and is read without I/O errors. Otherwise return None.
| Parameters: | sources (str or iterable(str)) – file path of the pak file to read, or an iterable specifying multiple such paths |
|---|---|
| Returns: | set of resource name strings if no read errors, None otherwise |
| Return type: | set(str) or None |
Example converter function that writes out the unmodified resource.
Treat all but the final path segments of the resource as subdirectories, and create them as needed. Then write the extracted resource out into the bottom subdirectory, using the final path segment of the resource name as the output file name.
In other words, if the name argument is “sound/hknight/grunt.wav”, then:
This function will always return True.
| Parameters: |
|
|---|---|
| Returns: | True |
| Return type: | bool |