Understanding the Difference in Golang and Python Zlib Outputs

When compressing a string using Zlib compression, Python's zlib library produces a different output compared to Golang's zlib implementation. Specifically, the fifth byte differs, with Python having a value of 0, while Golang has a value of 4.

Cause of the Difference

The disparity in outputs stems from the different flushing mechanisms used by the Python and Go libraries. Python's zlib defaults to Z_FLUSH, which flushes the buffer after compressing each block of data. In contrast, Golang's flate library, which implements Zlib, uses Z_SYNC_FLUSH by default. This behavior flushes the data after the entire input stream has been processed.

How to Get the Same Output in Golang

To obtain the same output as Python's zlib, replace Close() with Flush() in the Go code:

func compress(source string) []byte {
    buf := new(bytes.Buffer)
    w, _ := flate.NewWriter(buf, 7)
    w.Write([]byte(source))
    w.Flush()

    return buf.Bytes()
}

Bytes vs. Complete Stream

It's important to note that the output from the Python example is not a complete stream. It only flushes the buffer after compressing the first string.

Limitations of Byte-to-Byte Matching

Comparing the byte-to-byte output of different compression libraries to match compressed data is generally not feasible or practical. The output produced by compression libraries is guaranteed to be compatible, not identical.