top of page

Diagnostics (for technical support)

Usage

 

As command line options for genozip (Z), genounzip (U), genocat (C), genols (L)

 

Note: When used with genocat most options show only the requested metadata and not the file data itself.

 

Memory consumption

 

--show-memory[=PEAK]  

ZUCL. Show which Buffers are consuming the most memory. Normally, memory is sampled at the end of comprssion or decompression. With =PEAK, each Buffer retains its maximum allocation throughout execution.

kill -USR1 pid 

ZUCL. Executes --show-memory on a running process. Not available on Windows.

--debug-memory[=bytes] 

ZUCL. Show Buffer allocations and destructions. If <bytes> is specified then show only allocations of at least <bytes>.

--show-hash  

Z. See raw numbers that feed into determining the size of the global hash tables.

genozip file contents

 

--show-alleles  

ZUC. (VCF only) Output allele values to stdout. Each row corresponds to a row in the VCF file. Mixed-ploidy regions are padded and 2-digit allele values are replaced by an ascii character.

--show-dict[=field]

ZUC. Show dictionaries read/written for each vblock. With optional field (use --STATS to see the field names in the file) shows only that one field.
 

--show-counts=field

ZUC. Show (per snip in dictionary) the number of words in the file using this snip. genozip - works for any context (use --STATS to see context names). genounzip/genocat - works only for contexts that have a SEC_COUNTS section (which include 

any contexts in a file generated with genozip --show-counts of that context).

--show-b250[=field]

ZUC. Show b250 sections content - each value shows the line (counting from 1) and the index into its dictionary (note: REF 

and ALT are compressed together as they are correlated). With optional field (eg CHROM ; RNAME ; POS ; AN etc) shows only

that one field. This also works with genounzip and genocat but without the line numbers.

--dump-b250=field

.ZUC.  Dump the binary content of the b250 data of this field exactly as they appear in the genozip format to a file named 

"field.b250" - specify the field name as it appears in the Name column in --STATS for fields that have "comp b250" data.

--dump-local=field  

ZUC.  Same as --dump-b250 just for the local buffer.

--contigs

ZUC.  List the names of the chromosomes (or contigs) included in the file. Alternative names: --chroms --list-chroms

--dump-section section-type

ZUC. Dump the uncompressed unencrypted contents of all sections of this type (as it appears in 

--show-gheaders eg SEC_REFERENCE) to a files named "section-type.vb.dict_id.[header|body]".

 

--show-headers section-type

Show all the sections headers or those of a specific section type if the optional argument is provided. Argument is a case-insesitive substring of a section name. genozip and genounzip show the headers encountered in their normal operation, 

while genocat shows all the headers in the file, in the order they are in the file.

--force-gencomp

Z. SAM/BAM: produce PRIM/DEPN components even for non-sorted files.

--no-gencomp

Z. SAM/BAM: don't produce PRIM/DEPN components.

--no-domqual

Z. SAM/BAM/FASTQ: don't use the DOMQUAL codec when compressing QUAL.

--show-index  

ZUC. Show the content of the random access index (SEC_RANDOM_ACCESS section).

--show-lines  

Z. Show the byte offset of each line

--show-reference  

ZUC. Show the ranges included the SEC_REFERENCE sections

--show-ranges  

UC. Show the ranges as in RefStruct.ranges

--show-ref-seq  

ZUC. Show the reference sequences as stored internally in a SAM BAM or FASTQ file (also works for a reference file but 

--reference --regions is faster). Combine with --regions to see specific regions (genocat only). Combine with 

--sequential to omit newlines. '-' appears in unset loci.

--show-ref-diff  

C. Show the difference between two reference files. Use in combination with two --reference arguments.

--show-ref-index  

ZUC. Show the content of the random access index of the reference data (SEC_REF_RAND_ACC section).

--show-ref-hash  

ZUC. Show the details of the reference hash table (SEC_REF_HASH) sections.

--show-chrom2ref  

ZUC. Show the details of the file contigs that are mapped to a different contig name in the reference (eg '22' ➔ 'chr22').

--show-ref-contigs  

ZUC. Show the details of the reference contigs.

--show-ref-iupacs  

ZC. Show the the IUPACs in the reference. In combination with 

genozip --chain - also shows the VCF variants that have a IUPAC in the Luft reference and how they are handled.

--show-kraken  

C. Show inclusion or exclusion of lines. Used in combination with --taxid.

--show-txt-contigs  

ZUC. (SAM and BAM) Show the details of the contigs appearing the file header (SQ lines).

--show-gheader  

ZUC.  Show the content of the genozip header (which also includes the list of all sections in the file). 

--show-gheader=2 shows the section list after modification (if any) by writer_create_plan.

--show-vblocks

ZUC.  Show vblock headers as they are read / written.

--show-aliases  

ZUC. See contents of SEC_DICT_ID_ALIASES section.

--show-reference  

ZUC. Show the ranges included the SEC_REFERENCE sections.

--show-is-set contig

UC. Shows the contents of SEC_REF_IS_SET section for contig.

--show-bgzf  

ZUC. Show BGZF blocks as they are being compressed or decompressed.

--show-sag[=grp_i]  

ZUC. SAM/BAM: Show SA groups (supplementary / secondary alignments + their primary alignment).

--show-depn  

Z. SAM/BAM: Show supplementary / secondary alignments that are successfully mapped against a primary alignment.

--show-dvcf  

C. Show line-by-line outcome of the liftover of that line. Used with dual-coordinate files and may be combined --luft.

See: Dual-coordinate VCF files

 

Text file contents

 

--show-bam  

C. Show alignments of a BAM file.

Subsetting a file for debugging

 

--biopsy=vb[,vb...]

Z. Dump a subset VBs of the source file being compressed and including the txt header. The argument is a comma separated list of VB numbers or VB ranges. An argument of 0 means txt header only.

 

Examplegenozip mybam.bam --biopsy 5-7,11 will emit the txt header and VBs 5,6,7,11.

Notes: The biopsy is taken after reading the VBlocks without segging.

--biopsy-line=vb/line  

Z. Dump a single line. vb is 1-based VBlock number and line is 0-based line within the VBlock.

 

-B--vblock  

C. Use with a 'B' suffix to specify a low number of bytes eg -B100000B. Useful for then subsetting with --biopsy.

--head[=N]  

C. Compress only the first N lines (default: 10). When using this option Genozip compresses only VB=1 so vblock needs to 

be large enough to contain the specified number of lines.

 

Tracking execution

 

--show-containers[=field] or [=vblock_i]  

ZUC. Show flow of containers. Possibly with the values of a specific vblock_i or specific field (use 

--STATS to see the field names in the file).

--show-plan  

ZUC. Shows reconstruction plan. Combine with --luft to see Luft reconstruction plan.

--show-threads

ZUC.  Show thread dispatcher activity.

--debug-threads

ZUCL.  Alternative to --show-threads - store thread log in a buffer and display it in case of an error.

--debug-lines  

Z.  ZIP: adds an Adler32 signature to each line which will be verified in PIZ.

--add-line-numbers  

C.  SAM only: adds a field VB:Z describing the comp_i vblock_i and line_i of the line

--debug-seg[=field]  

Z.  Shows snips being segmented into contexts - possibly limiting to a specific field (use 

--STATS to see the field names in the file).

--count=VB  CU. 

Show number of lines written for each VBlock (note: --count without an argument shows lines written in the entire file).

--debug-progress

ZUC. See raw numbers that feed into the progress indicator.

--debug-stats  

Z. See details in the creation process of the --stats report.

--debug-generate  

Z. See contexts that are marked as "all the same" and are removed or shrunk.

--debug-recon-size  

Z. See vb->context[]->txt_len and vb->recon_size.

--debug-gencomp  

Z. SAM/BAM/VCF: View the queues of generated component buffers.

--debug-sag  

Z. SAM/BAM: For each failing candidate line for SA Groups - show the reason for its failure to get included.

--show-time[=res]

ZUCL. Show what functions are consuming the most time. Optional res is one of the members of ProfilerRec defined 

in profiler.h such 'compressor_lzma' or a substring such as 'compressor_'.

--show-aligner

ZUC. SAM/BAM/FASTQ: Show alignments of reads as generated by the Genozip aligner.

--show-digest  

ZUC. Show digest (MD5 or Adler32) updates.

--log-digest  

ZUC. Output the data hashed for digest_ctx_bound to digest.zip.log and digest.piz.log

--show-mutex[=mutex-name].  

ZUC. Shows locks and unlocks of all mutexes or a particular mutex.

--debug-read-ctxs

ZUC. Show all B250 LOCAL and DICT sections as they are read/skipped and decompressed during genocat/genounzip. 

Note: For genozip this is only relevant for reading sections of the first FASTQ file when compressing the second FASTQ 

file with --pair.

--show-uncompress

ZUC. Shows uncompressing of section data.

--debug-peek

UC. Shows reconstructor peek stack.

 

--show-flags

ZUCL. Shows internal flags after initialization.

--show-recon

UC. Shows the reconstruction plan.

--show-dvcf

C. When used with dual-coordinate VCF file shows for each variant its Coordinates (Primary Luft or Both) and its oStatus.

--show-wrong-MD

C. SAM/BAM with MD:Z field - shows cases where the special MD algorithm is not applied to the MD:Z in the data.

--show-wrong-XG

C. SAM/BAM with BS-Seeker2 XG:Z field - shows cases where the special XG algorithm is not applied to the XG:Z in the data.

--show-wrong-XM

C. SAM/BAM with Bismark or BSSeeker2 XM:Z field - shows cases where the special XM algorithm is not applied to the XM:Z in the data.

--show-wrong-XB

C. SAM with BSBolt XB:Z field - shows cases in which the predicted methylation string differs than the actual.

--debug-LONG

Z. SAM/BAM and FASTQ: treat data as long reads regardless of the actual read length.

--show-qual

ZUC. SAM/BAM and FASTQ: see internal data of the QUAL compression codecs.

--debug-qname

C. SAM/BAM and FASTQ: show QNAME flavor unit test.

--show-buddy

ZUC. SAM/BAM: show buddy (which can be a mate or saggy or both) for each line that has one.

Tracking compression performance

 

-w--stats   

Show the internal structure of a genozip file and the associated compression statistics.

-W--STATS   

Show more detailed statistics.

 

Note: specifying -W or -w twice, results in the header line of the statistics printed to stderr, thereby surviving piping stdout to grep

--show-filename

Show the file name for each file.

 

--stats=comp_i, --STATS=comp_i 

Z. similar to --stats or --STATS but shows stats for a single component

--show-codec  

Z. Genozip tests for the best codec when it first encounters a new type of data. See the results.

--verify-codec 

ZUC. Verifies each section's decompression correctness against an Adler32 that is stored in SectionHeader.magic. 

Note: the Genozip file generated when using this option is not a valid Genozip file as it has the wrong magic - 

this option is designed for detecting issues while developing new codecs.

 

Examplegenozip -t --verify-codec myfile.sam

 

--submit-stats  

Z. Submit aggregate stats of compression performance and metadata to the server

--debug-submit

          Z. Submits stats for debugging

--debug-debug

          ZUCL. Ad hoc option to assist debugging

Controlling execution

 

--one-vb vb  

UC. Reconstruct data from a single VB. Can be used with (1) genocat or (2) genounzip --test.

--seg-only  

Z. Run the segmenter but don't compress and don't write the output.

 

--xthreads  

ZUC. Use only one thread for the main PIZ/ZIP dispatcher. This doesn't affect thread use of other dispatchers.

--debug-latest  

ZU. Force genozip version upgrade notice

--debug  

ZUCL. Execute various debugging logic

bottom of page