MHash-384 is a fast, portable and secure hashing library, released under the MIT license.
The MHash-384 core library has been written in plain C, and the CLI front-end has been written in C++. The core library provides a simple "stream processing" API, that is available in two flavors: a plain C99 version and an object-oriented C++ wrapper. Either way, the MHash-384 library produces hash values with a fixed length of 384 bits (48 bytes).
MHash-384 supports a wide range of compilers, including MSVC++, GCC, MinGW/Cygwin, Clang/LLVM and Intel C++. It also runs on many platforms, including Windows, Linux, BSD and Solaris. Furthermore, the MHash-384 library has already been ported to various other programming languages, including Java, Microsoft.NET, Python as well as Delphi.
In order to use the MHash-384 library, simply include the header file mhash384.h in your code:
#include <mhash384.h>If you source code is written in C, use the mhash384_t struct and the mhash384_XYZ() functions:
/*variables*/
uint8_t buffer[BUFFSIZE];
size_t len;
uint8_t result[MHASH384_SIZE];
mhash384_t mhash384;
/*initialization*/
mhash384_init(&mhash384);
/*input data processing*/
while(more_data())
{
len = read_data(buffer, BUFFSIZE);
mhash384_update(&mhash384, buffer, len);
}
/*finalization*/
mhash384_final(&mhash384, result);Or, if you source code is written in C++, use the provided MHash384 wrapper class:
/*instance*/
MHash384 mhash384;
/*input data processing*/
while(more_data())
{
const std::vector<uint8_t> buffer = read_data();
mhash384.update(buffer);
}
/*finalization*/
const uint8_t *result = mhash384.finish();MHash-384 comes with a simple "standalone" command-line application, similar to the sha1sum orsha256sum utilities.
The MHash-384 command-line application takes a number of optional options followed by an one or more input files.
If no input file is specified, input will be read from standard input (stdin).
The digest will be written to the standard output (stdout). Diagnostic message are written to the standard error (stderr).
mhash384 [OPTIONS] [<FILE_1> <FILE_2> ... <FILE_N>]MHash-384 supports the following options:
--keep-going
Try to proceed with the remaining files, if a file has failed (e.g. access denied or file not found).
Only applicable, if multiple files have been specified. Default behavior is to abort immediately when a file has failed.
--short
Print the digest in short format (no file names). Default format prints the digest followed by the file name.
--lower-case
Print the digest in lower-case letters. Default format prints the digest in upper-case letters.
This option can only be used with the default Hex (hexadecimal) output format.
--base64
Print the digest in Base64 (RFC-4648) format. Default prints the digest in Hex (hexadecimal) output format.
This option must not be combined with the --base85 option, for obvious reasons.
--base85
Print the digest in Base85 (Ascii85) format. Default prints the digest in Hex (hexadecimal) output format.
This option must not be combined with the --base64 option, for obvious reasons.
--help
Print the help screen (manpage) and exit program.
--version
Print the program version (plus some build information) and exit program.
--self-test
Run self-test and exit program. This will process various standard test vectors and validate the resulting hashes.
Note: Some test vectors contain very long inputs, therefore the computation can take a while to complete!
--stress
Enable stress test mode. This will process all test strings from the specified input file, expecting one string per line.
All computed hash values are added to an std::unordered_set, thus checking for possible collisions.
--benchmark
Measure the overall time required for the operation. If specified, output the total amount of time elapsed, in seconds.
In default operation mode, MHash-384 writes one line per input file to the standard output:
<HASH_VALUE> [SPACE SPACE <FILE_NAME>]The format of the hash value is either Hex (hexadecimal) or Base64 (RFC-4648), depending on the specified options.
Also, the file name will be printed, unless "short" format was requested. File names may contain a path!
BD41A203A61FE74178A8D507...33E553FD1569ED733C52BE8B debian-7.9.0-amd64-DVD-1.iso
EE328DDD4E116165252F1FF8...11729801097C51FB61D20184 debian-7.9.0-i386-DVD-1.iso
A8B2007537867BDA0C18A264...45A1379AB8B4A77F9D8C8B24 debian-10.0.0-amd64-DVD-1.isoOn success, zero is returned. On error or user interruption, a non-zero error code is returned.
Note that, with "keep going" mode enabled, the exit code reflects whether at least one file was processed successfully.
Compute MHash-384 hash of a single file:
mhash384 "C:\Images\debian-8.3.0-amd64-DVD-1.iso"Compute MHash-384 hash of two files:
mhash384 "C:\Images\debian-8.3.0-amd64-DVD-1.iso" "C:\Images\debian-8.3.0-i386-DVD-1.iso"Compute MHash-384 hash of multiple files, using wildcard expansion (globbing):
mhash384 "C:\Images\*.iso"Compute MHash-384 hash from data passed directly via pipeline:
dd if=/dev/urandom bs=100 count=1 | mhash384Global definitions for both, the C and C++, API's.
The size of the final MHash-384 hash value (digest), in bytes. This value is qual to 48U.
The number of words per MHash-384 hash. Each word has a size of 64 bits (uint64_t). This value is qual to 6U.
All functions described in the following are reentrant and thread-safe. A single thread may compute multiple MHash-384 hashes in an "interleaved" fashion, provided that a separate MHash-384 context is used for each ongoing hash computation. Multiple threads may compute multiple MHash-384 hashes in parallel, provided that each thread uses its own separate MHash-384 context; no synchronization is required. However, sharing the same MHash-384 context between multiple threads is not safe in the general case. If the same MHash-384 context needs to be accessed from multiple threads, then the threads need to be synchronized explicitly (e.g. via Mutex lock), ensuring that all access to the shared context is rigorously serialized!
typedef struct mhash384_t;The MHash-384 context. It represents all state of an ongoing MHash-384 hash computation. Each MHash-384 hash computation needs a corresponding MHash-384 context. It is possible to re-use an MHash-384 context for multiple MHash-384 hash computations, provided that those hash computations are strictly serialized. If multiple MHash-384 hash computations need to be performed in an "interleaved" fashion, each ongoing hash computation needs to use its own separate MHash-384 context. In any case, the memory for the mhash384_t instance(s) must be allocated by the calling application. If the MHash-384 context was allocated on the heap space, the calling application also is responsible for freeing up that memory.
Note: Applications should treat this data-type as opaque, i.e. the application must not access the fields of the struct directly!
void mhash384_init(mhash384_t *const ctx);Set up the MHash-384 hash computation. This function initializes the MHash-384 context; it prepares the state for the upcoming hash computation. The application is required to call this function once for each MHash-384 hash computation. The function must to be called before any input data can be processed in a specific MHash-384 context! The application may call this function again, on the same MHash-384 context, which will reset that context and start a new hash computation.
Parameters:
mhash384_t *ctxmhash384_t that will be initialized (reset) by this operation.void mhash384_update(mhash384_t *const ctx, const uint8_t *const data_in, const size_t len);Process next chunk of input data. This function performs the actual MHash-384 hash computation, in an incremental way. The function processes the next N bytes of input data and updates the MHash-384 context (mhash384_t) accordingly. The application is supposed to call this function in a loop, with the same MHash-384 context, until all input has been processed.
Parameters:
mhash384_t *ctx
Pointer to the hash computation state of type mhash384_t that will be updated by this operation.
const uint8_t *data_in
Pointer to the input data to be processed by this operation. The input data needs to be located in one continuous block of memory. The given pointer specifies the base address, i.e. the address of the first byte to be processed.
Note: Formally, the input data is defined as an array of byte (uint8_t). Nonetheless, any kind of input data can be processed, by applying the proper typecast operator. For numeric values, the platform's endianness applies!
size_t len
The length of the input data to be processed, in bytes. Specify sizeof(T) * count for data types T other than byte.
Note: All bytes in the range from data_in[0] up to and including data_in[len-1] will be processed as input.
void mhash384_final(mhash384_t *const ctx, uint8_t *const digest_out);Retrieve final hash value. This function completes the MHash-384 hash computation and returns the computed hash value. The function finalizes the MHash-384 context (mhash384_t) and writes the resulting hash value to the output buffer. Once this function has been called, the corresponding MHash-384 context will be in an undefined state, until it is reset!
Parameters:
mhash384_t *ctx
Pointer to the hash computation state of type mhash384_t that will be finalized by this operation.
Note: The MHash-384 library does not free this memory; it may need to be freed up by the calling application!
uint8_t *digest_out
Pointer to the memory block where the final MHash-384 hash (digest) is to be stored. This memory needs to be allocated by the calling application! The size of the MHash-384 hash value, in bytes, is equal to MHASH384_SIZE.
Note: All bytes ranging from digest_out[0] up to and including digest_out[MHASH384_SIZE-1] will be overwritten!
void mhash384_compute(uint8_t *const digest_out, const uint8_t *const data_in, const size_t len);Compute hash value at once. This is a convenience function that can be used to compute an MHash-384 hash value with just a single invocation. The function processes a block of N input bytes and writes the resulting hash value to the output buffer. This function does not required the caller to provide an MHash-384 context; it internally uses a "transient" context. Anyway, this function is fully thread-safe. Naturally, this function is only applicable where all input data is available at once.
Parameters:
uint8_t *digest_out
Pointer to the memory block where the final MHash-384 hash (digest) is to be stored. This memory needs to be allocated by the calling application! This size of the MHash-384 hash value, in bytes, is equal to MHASH384_SIZE.
Note: All bytes ranging from digest_out[0] up to and including digest_out[MHASH384_SIZE-1] will be overwritten!
const uint8_t *data_in
Pointer to the input data to be processed by this operation. The input data needs to be located in one continuous block of memory. The given pointer specifies the base address, i.e. the address of the first byte to be processed.
Note: Formally, the input data is defined as an array of byte (uint8_t). Nonetheless, any kind of input data can be processed, by applying the proper typecast operator. For numeric values, the platform's endianness applies!
size_t len
The length of the input data to be processed, in bytes. Specify sizeof(T) * count for data types T other than byte.
Note: All bytes in the range from data_in[0] up to and including data_in[len-1] will be processed as input.
void mhash384_version (uint16_t *const major, uint16_t *const minor, uint16_t *const patch);Retrieve version information. This function returns the current version of the MHash-384 library.
Parameters:
uint16_t *major
Pointer to a variable of type uint16_t where the major version of the MHash-384 library will be stored.
uint16_t *minor
Pointer to a variable of type uint16_t where the minor version of the MHash-384 library will be stored.
uint16_t *patch
Pointer to a variable of type uint16_t where the patch level of the MHash-384 library will be stored.
bool mhash384_selftest(void);Self-test routine. This function runs the built-in self-test of the MHash-384 library; intended for debugging purposes.
Return value:
true, if the self-test completed successfully; returns false, if any problems have been detected.For the C++ langauge, the MHash384 class is provided, as a convenience wrapper around the C-API. All functions of the MHash384 class are reentrant and thread-safe. A single thread may compute multiple MHash-384 hashes in an "interleaved" fashion, provided that a separate MHash384 instance (object) is used for each ongoing hash computation. Multiple threads may compute multiple MHash-384 hashes in parallel, provided that each thread uses its own separate MHash384 instance; no synchronization is required. However, sharing the same MHash384 instance between multiple threads is not safe in the general case. If the same MHash384 instance needs to be accessed from multiple threads, then the threads need to be synchronized explicitly (e.g. via Mutex lock), ensuring that all access to the shared instance is rigorously serialized!
MHash384(void)Constructor. Creates a new MHash384 instance (object) and prepares the state for the upcoming hash computation. Each instance internally maintains the corresponding MHash-384 context. The application is required to create a separate MHash384 instance for each ongoing MHash-384 hash computation; it is possible to re-use an MHash384 instance for multiple MHash-384 hash computations, provided that those hash computations are strictly serialized.
Note: The application is required to allocate the memory for the MHash384 instance. If the instance was allocated on the heap (dynamic storage duration), the application is also required to explicitly destroy the instance, when no longer needed.
void MHash384::update(const std::uint8_t *const data, const size_t len)Process next chunk of input data. This function performs the actual MHash-384 hash computation, in an incremental way. The function processes the next N bytes of input data and updates the internal MHash-384 context accordingly. The application is supposed to call this function in a loop, on the same MHash384 instance, until all input has been processed.
Parameters:
const uint8_t *data_in
Pointer to the input data to be processed by this operation. The input data needs to be located in one continuous block of memory. The given pointer specifies the base address, i.e. the address of the first byte to be processed.
Note: Formally, the input data is defined as an array of byte (uint8_t). Nonetheless, any kind of input data can be processed, by applying the proper typecast operator. For numeric values, the platform's endianness applies!
size_t len
The length of the input data to be processed, in bytes. Specify sizeof(T) * count for data types T other than byte.
Note: All bytes in the range from data_in[0] up to and including data_in[len-1] will be processed as input.
template<size_t size>
void MHash384::update(const std::array<std::uint8_t, size> &data)A convenience overload of the MHash384::update() function, which processes an std::array<uint8_t, N> as input.
Parameters:
const std::array<uint8_t, N> &datastd::array<uint8_t, N> containing the input data to be processed.data[0] up to and including data[data.size()-1] will be processed as input.void MHash384::update(const std::vector<std::uint8_t> &data)A convenience overload of the MHash384::update() function, which processes an std::vector<uint8_t> as input.
Parameters:
const std::vector<std::uint8_t> &datastd::vector<uint8_t> containing the input data to be processed.data[0] up to and including data[data.size()-1] will be processed as input.void MHash384::update(const std::string &text)A convenience overload of the MHash384::update() function, which processes an std::string as input.
Parameters:
const std::string &textstd::string containing the input data to be processed.text[0] up to and including text[text.length()-1] will be processed as input. Each character in the std::string is processed as a byte value, disregarding any specific character encoding.void MHash384::update(const char *const text)A convenience overload of the MHash384::update() function, which processes a NULL-terminated C string as input.
Parameters:
const char *const texttext[0] up to and including text[strlen(text)-1] will be processed as input. Each character in the C string is processed as a byte value, disregarding any specific character encoding.template<typename element_type>
void MHash384::update(const element_type *const address);A convenience overload of the MHash384::update() function, which processes an object designated by a pointer.
Parameters:
const element_type *const addressaddress[0] up to and including address[sizeof(element_type)-1] will be processed as input.template<typename element_type>
void MHash384::update(const element_type &element);A convenience overload of the MHash384::update() function, which processes an object designated by a reference.
Parameters:
const element_type &elementaddr[0] up to and including addr[sizeof(element_type)-1] with addr = std::addressof(element) will be processed as input.template<typename iterator_type>
void MHash384::update(const iterator_type &first, const iterator_type &last)A convenience overload of the MHash384::update() function, which processes a sequence of elements via iterators.
Parameters:
const iterator_type &first
Read-only reference to the iterator designating the first element to be processed.
Note: All elements in the range from *first up to but excluding *last will be processed as input. Each element in this range is processed as a byte-sequence, like a POD, assuming a size of sizeof(iterator_type::value_type).
const iterator_type &last
Read-only reference to the iterator designating the element just after the last element to be processed.
Note: All elements in the range from *first up to but excluding *last will be processed as input. Each element in this range is processed as a byte-sequence, like a POD, assuming a size of sizeof(iterator_type::value_type).
const std::uint8_t *MHash384::finish(void)Retrieve final hash value. This function completes the MHash-384 hash computation. The function finalizes the internal MHash-384 context, if it was not finalized yet, and returns a pointer to the buffer containing the resulting hash value. Once this function has been called, the MHash384 instance remains in the finalized state, until it is reset.
Warning: Trying to process more input data while the MHash384 instance is in finalized state will throw an exception!
Return value:
MHash384 instance. The size of the MHash-384 hash value, in bytes, is equal to MHASH384_SIZE.MHash384 instance is reset or destroyed. If the hash value needs to be retained after the instance was reset/destroyed, the application must copy the hash value to a separate buffer!void MHash384::reset(void)Reset the MHash-384 hash computation. This function re-initializes the internal MHash-384 context, thus starting a new MHash-384 hash computation. It is not necessary to explicitly call this function on a new MHash384 instance; it is called implicitly by the constructor. However, it is possible to re-use an existing MHash384 instance for multiple (strictly serialized) MHash-384 hash computations, by calling this function in between each pair of consecutive hash computations.
MHash-384 has been tested to successfully build and run on (at least) the following platforms:
The MHash-384 C/C++ library and CLI front-end can be built using (at least) one of the following build systems:
MHash-384 can be built for the Windows platform using the Microsoft Visual C++ compiler, version 16.00 or later.
The provided project/solution files should build successfully with Visual Studio 2010 or later. However, be aware that it may be necessary to adjust the "Platform Toolset" to your specific version of Visual Studio in all projects! Build configurations are available for both, 32-Bit (Win32) and 64-Bit (x64) Windows, but the 64-Bit flavor is recommended for best performance.
Note: You can download the latest version of the Visual Studio "Community" edition for free from the official web-site.
Building MHash-384 from the developer command prompt is supported via the MSBuild tool:
MSBuild.exe /property:Configuration=Release /property:Platform=x64 /target:Rebuild "MHash384.sln"MHash-384 can be built using the GNU C/C++ compiler (GCC), version 4.8 or later, or any GCC-compatible compiler, such as Clang/LLVM, on a wide range of platforms; supported platforms include Linux, the BSD family, Solaris and Windows.
The provided makefiles should build successfully with GNUmake on any supported platform. GNUmake is the default make implementation on Linux/GNU, but may need to be installed separately and invoked as gmake on BSD and Solaris. GCC or a GCC-compatible compiler (e.g. Clang/LLVM) is available out-of-the-box on most supported platforms; otherwise it can usually be installed from the system's package manager. Please see the documentation of your specific distribution for details!
In order to build MHash-384, simply run make from the MHash-384 base directory, for example:
$ make -B MARCH=x86-64 MTUNE=intel STATIC=1The following options can be used to tweak the behavior of the provided makefiles:
MARCH: Generate machine code for the specified CPU type, see -march for details (default is native)MTUNE: Tune the generated machine code for the specified CPU type, see -mtune for details (default is native)STATIC: If set to 1, link with static CRT libraries; otherwise link with shared CRT libraries (default is 0)DEBUG: If set to 1, generate a binary suitable for debugging; otherwise generate an optimized binary (default is 0)NODOCS: If set to 1, the HTML documents are no generated; useful where pandoc is unavailable (default is 0)SANITIZE: Instrument the binary with the specified sanitizer, e.g. address to enable the AddressSanitizer (no default)The following options can be used to override the default tools used by the makefiles:
CXX: The C++ compiler to be used (default is system-specific, e.g. g++ or clang++)AR: The archiver to be used (default is system-specific, usually ar)STRIP: The strip program to be used (default is strip)PNDC: The document converter to be used (default is pandoc)TAR: The tarball program to be used (default is tar)WNDRS: The Windows resource compiler to be used, used on Cygwin and MinGW only (default is windres)ZIP: The zip program to be used, used on Cygwin and MinGW only (default is zip)It is possible to build MHash-384 with GCC or Clang/LLVM on the Windows platform thanks to Cygwin or MinGW/MSYS. However, if you want to build with GCC or Clang/LLVM on Windows nowadays, then it is highly recommended to use MSYS2 in conjunction with Mingw-w64 – even for 32-Bit targets! The “old” Mingw.org (Mingw32) project is considered deprecated.
Just follow the basic MSYS2 setup procedure, as described on the official web-site, then install Mingw-w64:
pacman -S base-devel mingw-w64-i686-toolchain mingw-w64-x86_64-toolchainThis section contains a pseudo-code description of the MHash-384 algorithm:
Pre-defined constants for MHash-384 computation:
const
MHASH384_SIZE := 48 /*size of the hash, in bytes*/
MHASH384_WORDS := 6 /*size of the hash, in 64-Bit words*/
MHASH384_INI: array[0..MHASH384_WORDS-1] of UInt64 /*the initial state vector*/
MHASH384_FIN: array[0..MHASH384_SIZE-1] of Byte /*byte indices for the finalization routine*/
MHASH384_XOR: array[0..256, 0..MHASH384_WORDS-1] of UInt64 /*LUT for XOR (exclusive or) constants*/
MHASH384_ADD: array[0..256, 0..MHASH384_WORDS-1] of UInt64 /*LUT for ADD (arithmetic addition) constants*/
MHASH384_MIX: array[0..255, 0..MHASH384_WORDS-1] of Byte /*LUT containing the "mixing" indices*/Note: The lookup tables MHASH384_XOR and MHASH384_ADD have been pre-computed in such a way that each of the 257 rows (each with a size of 48 Bytes) has a hamming distance of at least 182 bits to any other row. This ensures that, for each possible value an input byte can take, a different set of state bits will be "flipped" by the XOR (exclusive or) operation.
The state of an ongoing MHash-384 computation:
type MHash384State = record
rnd: UInt8
hash: array[0..MHASH384_WORDS-1] of UInt64Set up the MHash-384 state for a new hash computation:
procedure MHash384_Initialize
state.rnd ← 0
state.hash ← MHASH384_INIUpdate the MHash-384 state with the next N input (message) bytes:
procedure MHash384_Update
input:
message: array[0..N-1] of Byte
for each Byte b in message do
_MHash364_Iterate(MHASH384_XOR[b], MHASH384_ADD[b], MHASH384_MIX[rnd])
state.rnd ← (state.rnd + 1) mod 256Note: This routine can be invoked multiple times in order to process in the input message in "chunks" of arbitrary size.
Compute the final hash value (digest), once all input has been processed:
procedure MHash384_Update
var:
previous: UInt16
output:
digest: array[0..MHASH384_SIZE-1] of Byte
previous ← 256;
for i = 0 to HASH384_SIZE-1 do
_MHash364_Iterate(MHASH384_XOR[previous], MHASH384_ADD[previous], MHASH384_MIX[rnd])
state.rnd ← (state.rnd + 1) mod 256
previous ← (digest[i] ← _MHash384_GetByte(MHASH384_FIN[i]))Internal processing routine, used by the "update" and "finalization" routines:
procedure _MHash364_Iterate
var:
temp: array[0..MHASH384_WORDS-1] of UInt64
input:
xor_row: array[0..MHASH384_WORDS-1] of UInt64
add_row: array[0..MHASH384_WORDS-1] of UInt64
mix_row: array[0..MHASH384_WORDS-1] of Byte
for i = 0 to HASH384_WORDS-1 do
temp[i] ← Hash128to64(state.hash[i] + add_row[i], state.hash[mix_row[i]]) ⊻ xor_row[i]
state.hash ← tempNote: Here the ⊻ symbol denotes the bit-wise XOR (exclusive or) operator. Furthermore, the Hash128to64() routine is adopted from the function of the same name that appears in Google's CityHash. Please see here for details!
Internal routine to extract a specific byte from the current state:
procedure _MHash384_GetByte
input:
index: Byte
output:
value: Byte
value ← (state.hash[index ÷ 8] ≫ ((index mod 8) × 8)) mod 256Note: Here the ÷ symbol denotes integer division, i.e. an arithmetic division in which the fractional part (remainder) is discarded. Furthermore, the ≫ symbol denotes the bit-wise "right shift" operator (shift bits to the right by n places).
MHash-384 - Simple fast portable secure hashing library
© 2016-2020 LoRd_MuldeR
Permission is hereby granted, free of charge, to any person obtaining a copy of this software
and associated documentation files (the "Software"), to deal in the Software without
restriction, including without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or
substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.https://opensource.org/licenses/MIT