Early WIP on v2.55

[Michael Lobko-Lobanovsky]

... (the same to what i am doing in WinLIFT when skinning ListView).

BTW I'm using SendMessage(WM_SETREDRAW FALSE/TRUE) for both ListViews when populating them at model load time. Otherwise their scrollbars flicker like hell unskinned in the process.

The RTF PowerBASIC project i sent you, was designed to work on Seven (written in 2016 by José Roca), thus i do not understand why it doesn't work by you 

It does work for me under W7 but exactly the way any other genuine WordPad pages work: with indexed colors and image dither. If an ARGB PNG is embedded in the page markup manually, then its background (substrate) color is preserved but the foreground colors still get dithered and the image sizes on the page get broken to invisibility.

Evidently José Roca has no authority on my workstation despite all his numerous services to the BASIC Vaterland. 

[Patrice Terrier]

I have almost completed my new Scrollbar.h to skin any control scrollbar on the fly.
That was a tedious work because i had to remember what i have done in WinLIFT, to add new support for the RICHEDIT control.
I have exported several WinLIFT internal functions to retrieve the bitmap handle of the scrollbar bitmap components.

[Michael Lobko-Lobanovsky]

Amazing!

IIRC this is something I can't do yet.

[Patrice Terrier]

Here is an example of the reason why i am always using the latest API and the current OS

The compress API is not available on Seven
https://docs.microsoft.com/en-us/windows/desktop/cmpapi/-compression-portal
this is what i wanted to use with my new help system to decrease the size of the .rtf files, and produce a lossless .ztf file that is almost half the size of the original when using embedded images.

Here is the link to the C++ compression/decompression examples to test in W10 console mode.
https://docs.microsoft.com/en-us/windows/desktop/cmpapi/using-the-compression-api-in-buffer-mode
and it is very fast…

Added
12-31-2018
I did several minor changes in OR, that means you will have to sync with me, once you get back from holidays.
I have completed the W10 help support, i am very pleased of the result, and the new skinned scrollbar API works like a charm with it. The rtf compression is not added yet, but it is top on my list, once i have completed the writing of the basic documentation.

01-01-2018
I have fixed an array bug in ResetCamera.
I have written the missing MENU_FILE_REOPEN option.
I have completed the basic help documentation.
I have worked on a new model

01-02-2019
And the whole combination of the new .ztf help files is only 808 Kb with all the embedded images!
Thanks to the W10 COMPRESS_ALGORITHM_XPRESS_HUFF that is amazing both in size and speed.
I have written two console utilities to compress/decompress the help files for the purpose of text edition (zCompress.exe and zDecompress.exe)
The attachment shows the new OR help skinned window on my W10 system (with mouse wheel support), using the new ScrollbarEx API control.

To compress a RTF file with zCompress, use this syntax from a prompt command line:
zCompress text.rtf (create automatically text.ztf into the same folder)

To decompress a ZTF file with zDeCompress, use this syntax from a prompt command line:
zDeCompress text.ztf (create automatically text.rtf into the same folder)

01-03-2019
Sync patch attached to this post…

[Patrice Terrier]

My friend,

To save your time, here is the uncompressed version of the rtf files, i forgot to put them in the previous patch.
They shouldn't be hard to convert to html, to let them work with HH on Seven.

I have also attached WinLIFT64 compiled with VS2010 for smaller binary size.

[Michael Lobko-Lobanovsky]

Howdy Patrice,

You did an impressive amount of work these last few days!

Thanks for the uncompressed files -- with your permission, I'll use them to generate an equivalent .CHM file for the HH help. I'd also like to have a precompiled ObjReader64.exe to be able to test your help system at all because I can't even compile my own one as it requires a W10 version of Windows SDK -- something that I'm not going to touch until there are still pre-W10 OS'es that Microsoft keeps supporting.

Tell you more, though W8/8.1 can deal with the MS compression API in principle, it still requires an extra non-system Cabinet.dll library to be installed on the user PC.

OTOH I found some open-source Linuxoid code pieces that, apart from other functionality, are claimed to be able to decompress MS XPRESS_HUFF data, and they even claim that it works twice faster than the original. All in all, the code pulls in a lot of Lin/Win dependencies and compiles only with GCC. But I think I can isolate the decompression code, strip it of 3rd party references, make it MS VC friendly, and supply it with a standard MS (de)compression interface to be used literally in place of your W10-only APIs. Then we could have a reasonably small decompression include file to be linked statically in OR to make your system user-friendly on the pre-W10 platforms.

What's your opinion on this possibility?

[Patrice Terrier]

Thanks for the uncompressed files -- with your permission, I'll use them to generate an equivalent .CHM file for the HH help.

Of course you have my permission, this is the reason why i sent you the uncompressed version.

I have attached the W10 ObjReader64 version compiled with VS2017, to let you try the ztf files in context.

Many of my latest projects (a.k.a. MBox64  and consor) are targeted to work with the latest API, and as much i disliked Windows 8, as much i am pleased of W10, that works very well even on my small ATOM Z8700 Surface (2 Gb RAM) that starts in less than 5 seconds with it!

I plan to test soon VS2019 to check the latest facilities provided by the new environment.

make it MS VC friendly, and supply it with a standard MS (de)compression interface to be used literally in place of your W10-only APIs

There is another API that could be used for compression/decompression, and it has been used by WinLIFT for years (see skSkiToDib).
And here is a link to the official documentation:
https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/content/ntifs/nf-ntifs-rtldecompressbuffer
Note: RtlCompressBuffer seems also to support COMPRESSION_FORMAT_XPRESS_HUFF.

Here is the complete WinLIFT encapsulation, that i could export and it should work on everything
At the time i started to use it, it was a NTAPI undocumented couple of functions, and my code keep linking directly to the core ntdll API, bypassing all other extra encapsulations that have been added since XP.

Code: [Select]

long RtlDecompressBuffer (IN WORD nFormat, OUT CHAR* &szUBUF, IN DWORD nSizeUBUF, IN CHAR* szCBUF, IN DWORD nSizeCBUF, OUT DWORD &FinalSize) {
    long nRet = 0;
    HMODULE hDll = LoadLibrary(L"ntdll.dll");
    if (hDll) {
        long_proc (WORD, CHAR*, DWORD, CHAR*, DWORD, DWORD*);
        zProc hProc = (zProc) GetProcAddress(hDll, "RtlDecompressBuffer");
        if (hProc) { nRet = hProc(nFormat, szUBUF, nSizeUBUF, szCBUF, nSizeCBUF, &FinalSize); }
    }
    return nRet;
}

long RtlGetCompressionWorkSpaceSize (IN USHORT nFormat, OUT DWORD &nSizeCBUF, OUT DWORD &nSizeWorkSpace) {
    long nRet = 0;
    HMODULE hDll = LoadLibrary(L"ntdll.dll");
    if (hDll) {
        long_proc (WORD, DWORD*, DWORD*);
        zProc hProc = (zProc) GetProcAddress(hDll, "RtlGetCompressionWorkSpaceSize");
        if (hProc) { nRet = hProc(nFormat, &nSizeCBUF, &nSizeWorkSpace); }
    }
    return nRet;
}

long RtlCompressBuffer (IN WORD nFormat, IN PUCHAR szUBUF, IN DWORD nSizeUBUF, OUT PUCHAR szCBUF, IN DWORD nSizeCBUF, IN DWORD nSizeChunk,
                        OUT DWORD &FinalSize, IN PVOID WorkSpace) {
    long nRet = 0;
    HMODULE hDll = LoadLibrary(L"ntdll.dll");
    if (hDll) {
        long_proc (WORD, PUCHAR, DWORD, PUCHAR, DWORD, DWORD, DWORD*, PVOID);
        zProc hProc = (zProc) GetProcAddress(hDll, "RtlCompressBuffer");
        if (hProc) { nRet = hProc(nFormat, szUBUF, nSizeUBUF, szCBUF, nSizeCBUF, nSizeChunk, &FinalSize, WorkSpace); }
    }
    return nRet;
}

HBITMAP skSkiToDib (IN WCHAR* szFile, OUT long &nReserved) { // dllexport
    HBITMAP nFunction = 0;
    HDC hIC = 0;
    HBITMAP hDIB = 0;
    if (szFile) {
        if (FileExist(szFile)) {
            HANDLE hFile = 0;
            if (zFOpen(szFile, 0, 2, hFile) == 0) {
                DWORD LenBufIn = zFlof(hFile);
                string sBuffer; sBuffer.assign(LenBufIn, 0);
                zFGet(hFile, sBuffer);
                zFClose(hFile);
                if (LenBufIn > 8) {
                    string sHeader = left$(sBuffer, 8);
                    string sMem;
                    short imgW = 0; sMem = mid$(sHeader, 1, 2); MoveMemory(&imgW, (CHAR*) sMem.c_str(), 2);
                    short imgH = 0; sMem = mid$(sHeader, 3, 2); MoveMemory(&imgH, (CHAR*) sMem.c_str(), 2);
                    nReserved = 0; sMem = mid$(sHeader, 5, 4); MoveMemory(&nReserved, (CHAR*) sMem.c_str(), 4);
       
                    if (g_nUnlocked) { // Detect if (skin are valid
                        char zC[] = {(CHAR) nReserved};
                        g_nUnlocked = instr(0, g_sSkinKey, zC);
                    }
       
                    hIC = skDisplayDC();
                    hDIB = skCreateDIBSection(hIC, imgW, imgH, 32);
                    if (hDIB) {
                        BITMAP bm; GetObject(hDIB, sizeof(bm), &bm);
       
                        LenBufIn -= 8;
                        sBuffer = mid$(sBuffer, 9, LenBufIn);
       
                        // Very important to use the good size for the sBufOut buffer
                        // or we may have a STATUS_BAD_COMPRESSION_BUFFER error!
                        DWORD LenBufOut = bm.bmWidthBytes * bm.bmHeight;
                        //string sBufOut = sBufOut.assign(LenBufOut, 0);
                        char* szBufOut = new char[LenBufOut];
                        memset(szBufOut, 0, LenBufOut);
                        DWORD cBuffSize = 0;
                        long nRet = RtlDecompressBuffer (COMPRESSION_FORMAT_LZNT1, szBufOut, LenBufOut, (CHAR*) sBuffer.c_str(), LenBufIn, cBuffSize);
                        MoveMemory(bm.bmBits, szBufOut, min(LenBufOut, cBuffSize));
                        delete [] szBufOut;
                        if (nRet == 0) { nFunction = hDIB; }
                    }
                    DeleteDC(hIC);
                }
            }
        }
    }
    return nFunction;
}

PS: Tell me if you need the Cabinet.dll

[Michael Lobko-Lobanovsky]

Thanks for the executable!

Here is the complete WinLIFT encapsulation, that i could export and it should work on everything

Would be useless as-is in this case because the decompressor that we need to re-create directly in ObjReader based on those exports would not support COMPRESSION_FORMAT_XPRESS_HUFF below MS Windows 8, and that's the format you're using in your ZTF help files. You'd have to also change your ZTF format to straight COMPRESSION_FORMAT_XPRESS to be compatible not only with W8/8.1/10 but also W7.

(How can you be so cold blooded and kill your app's compatibility with the potential sub-W10 user base with your own hands anyway? )

Tell me if you need the Cabinet.dll

No thanks, I'm not planning on testing OR under W8/8.1 at this point in time.

[Patrice Terrier]

How can you be so cold blooded and kill your app's compatibility with the potential sub-W10 user base with your own hands anyway?

Because here in France most peoples have moved fast to W10, and the private users even faster than the enterprises.
All my friends around me, have moved on, they were reluctant with 8 and 9 (8.1), but 10 has got all the suffrages.
See what ZDnet says about the french market
https://www.zdnet.fr/actualites/chiffres-cles-les-systemes-d-exploitation-sur-pc-39790131.htm

The only other large OS base, used here, is Android and i just bought on Amazon a new 8.4 HUAWEI Mediapad 5, that is a true jewel with its 2K screen.
https://consumer.huawei.com/fr/tablets/mediapad-m5/

IOS is still used by the bobos and the geeks, but the real big market here is W10.

I had the same problem at the time of VISTA, because i moved fast to DWM, while most people were still using XP, but that give me a big boost on my competitors because i learned how to use the new API(s) before all of them, that helped me to make a success of GDImage in the WinDev community.

Would be useless as-is in this case because the decompressor that we need to re-create directly in ObjReader based on those exports would not support COMPRESSION_FORMAT_XPRESS_HUFF below MS Windows 8,

I could use the same compression that is used in my .ski or my Fly audio files, that is working on XP+, or i could just use the rtf uncompressed as well
I shall make a size comparison of the two compression systems and will let you know the result.

[Michael Lobko-Lobanovsky]

https://www.zdnet.fr/actualites/chiffres-cles-les-systemes-d-exploitation-sur-pc-39790131.htm

But you can see yourself that every 3rd person in France is still using Windows 7! IMHO it is unwise to write them all off your list of potential users...

[Patrice Terrier]

I wrote a quick compressor project based on NTDLL for comparison purpose

Uncompressed file: 006Lighting.rtf 2150400 bytes

and the result are

with the W10 Cabinet.dll using COMPRESSION_FORMAT_XPRESS_HUFF the file is 184320 bytes

with NTDLL using COMPRESSION_FORMAT_XPRESS the file is 245760 bytes

with NTDLL using COMPRESSION_FORMAT_LZNT1 the file is 425984 bytes (smallest compression ratio, however the fastest)

[Michael Lobko-Lobanovsky]

Please let your message stay intact for comparison purposes (do not delete it). I'll see if I can do anything with the Linux code and COMPRESSION_FORMAT_XPRESS_HUFF decompression tomorrow.

[Patrice Terrier]

Just in case, here is the console code i wrote to compress the test file.
As you can see i could easily switch from one to another to select one that give the best result in the old and new Windows paradigm

Tools.h

Code: [Select]

#pragma once
using namespace std;

#define void_proc_c typedef void (__cdecl *zProc)
#define long_proc_c typedef long (__cdecl *zProc)
#define long_proc typedef long (__stdcall *zProc)

WCHAR* wcscopy (IN WCHAR* dst, IN const WCHAR* src) {
    // Warning, there is no check for buffer overflow !
    long nLen = lstrlen(src);
    //for (long K = 0; K < nLen; K++) {
    //    dst[K] = src[K];
    //}
    MoveMemory(dst, src, nLen * sizeof(WCHAR));
    dst[nLen] = L'\0';
    return dst;
}

HMODULE MSVCRT() {
    static HMODULE hModule;
    if (hModule == 0) { hModule = LoadLibrary(L"MSVCRT"); }
    return hModule;
}

void wsplitpath(IN WCHAR* FullPath, OUT WCHAR* Drive, OUT WCHAR* Dir, OUT WCHAR* FileName, OUT WCHAR* Ext) {
    HMODULE hModule = MSVCRT();
    if (hModule) {
        void_proc_c(WCHAR*, WCHAR*, WCHAR*, WCHAR*, WCHAR*);
        zProc hProc = (zProc)GetProcAddress(hModule, "_wsplitpath");
        if (hProc) { hProc(FullPath, Drive, Dir, FileName, Ext); }
    }
}

long Add_Str(OUT WCHAR* dest, IN WCHAR* srce) {
    long nRet = -1; // Error
    HMODULE hModule =  MSVCRT();
    if (hModule) {
        long_proc_c (WCHAR*, WCHAR*, size_t);
        zProc hProc = (zProc)GetProcAddress(hModule, "wcsncat");
        if (hProc) { nRet = hProc(dest, srce, _TRUNCATE); }
    }
    return nRet;
}

HMODULE NTDLL() {
    static HMODULE hModule;
    if (hModule == 0) { hModule = LoadLibrary(L"ntdll"); }
    return hModule;
}

long RtlDecompressBuffer (IN WORD nFormat, OUT CHAR* &szUBUF, IN DWORD nSizeUBUF, IN CHAR* szCBUF, IN DWORD nSizeCBUF, OUT DWORD &FinalSize) {
    long nRet = 0;
    HMODULE hModule =  NTDLL();
    if (hModule) {
        long_proc (WORD, CHAR*, DWORD, CHAR*, DWORD, DWORD*);
        zProc hProc = (zProc) GetProcAddress(hModule, "RtlDecompressBuffer");
        if (hProc) { nRet = hProc(nFormat, szUBUF, nSizeUBUF, szCBUF, nSizeCBUF, &FinalSize); }
    }
    return nRet;
}

long RtlGetCompressionWorkSpaceSize (IN USHORT nFormat, OUT DWORD &nSizeCBUF, OUT DWORD &nSizeWorkSpace) {
    long nRet = 0;
    HMODULE hModule =  NTDLL();
    if (hModule) {
        long_proc (WORD, DWORD*, DWORD*);
        zProc hProc = (zProc) GetProcAddress(hModule, "RtlGetCompressionWorkSpaceSize");
        if (hProc) { nRet = hProc(nFormat, &nSizeCBUF, &nSizeWorkSpace); }
    }
    return nRet;
}

long RtlCompressBuffer (IN WORD nFormat, IN PUCHAR szUBUF, IN DWORD nSizeUBUF, OUT PUCHAR szCBUF, IN DWORD nSizeCBUF, IN DWORD nSizeChunk,
                        OUT DWORD &FinalSize, IN PVOID WorkSpace) {
    long nRet = 0;
    HMODULE hModule =  NTDLL();
    if (hModule) {
        long_proc (WORD, PUCHAR, DWORD, PUCHAR, DWORD, DWORD, DWORD*, PVOID);
        zProc hProc = (zProc) GetProcAddress(hModule, "RtlCompressBuffer");
        if (hProc) { nRet = hProc(nFormat, szUBUF, nSizeUBUF, szCBUF, nSizeCBUF, nSizeChunk, &FinalSize, WorkSpace); }
    }
    return nRet;
}

and wmain

Code: [Select]

// zCompress.cpp : This file contains the 'main' function. Program execution begins and ends there.
#include <iostream>


// Run program: Ctrl + F5 or Debug > Start Without Debugging menu
// Debug program: F5 or Debug > Start Debugging menu

// Tips for Getting Started:
//   1. Use the Solution Explorer window to add/manage files
//   2. Use the Team Explorer window to connect to source control
//   3. Use the Output window to see build output and other messages
//   4. Use the Error List window to view errors
//   5. Go to Project > Add New Item to create new code files, or Project > Add Existing Item to add existing code files to the project
//   6. In the future, to open this project again, go to File > Open > Project and select the .sln file
#include <Windows.h>
#include <stdio.h>
#include <compressapi.h>
#include "Tools.h"

// Windows 10, Cabinet.dll version
// Warning, the COMPRESS_ALGORITHM_XPRESS_HUFF is not supported on older OS
//
//void wmain(_In_ int argc, _In_ WCHAR *argv[]) {
//    COMPRESSOR_HANDLE Compressor    = NULL;
//    PBYTE CompressedBuffer          = NULL;
//    PBYTE InputBuffer               = NULL;
//    HANDLE InputFile                = INVALID_HANDLE_VALUE;
//    HANDLE CompressedFile           = INVALID_HANDLE_VALUE;   
//    BOOL DeleteTargetFile           = TRUE;
//    BOOL Success;
//    SIZE_T CompressedDataSize, CompressedBufferSize;
//    DWORD InputFileSize, ByteRead, ByteWritten;
//    LARGE_INTEGER FileSize;   
//    ULONGLONG StartTime, EndTime;
//    double TimeDuration;
//
//    if (argc != 2) {
//        wprintf(L"Usage:\n\t%s <input_file_name>\n", argv[0]);
//        return;
//    }
//
//    WCHAR zFileIn[MAX_PATH] = { 0 }; wcscopy(zFileIn, argv[1]);
//    WCHAR zFileOut[MAX_PATH] = { 0 };
//    WCHAR drive[_MAX_DRIVE], dir[_MAX_DIR], fname[_MAX_FNAME], ext[_MAX_EXT];
//    wsplitpath(zFileIn, drive, dir, fname, ext);
//    if (wcsstr(L".rtf", ext) == 0) {
//        wprintf(L"The input file must be a valid .rtf file.\n");
//        goto done;
//    }
//
//    wcscopy(ext, L".ztf");
//    wcscopy(zFileOut, drive); Add_Str(zFileOut, dir); Add_Str(zFileOut, fname); Add_Str(zFileOut, ext);
//
//    //  Open input file for reading, existing file only.
//    InputFile = CreateFile(
//        zFileIn,                  //  Input file name
//        GENERIC_READ,             //  Open for reading
//        FILE_SHARE_READ,          //  Share for read
//        NULL,                     //  Default security
//        OPEN_EXISTING,            //  Existing file only
//        FILE_ATTRIBUTE_NORMAL,    //  Normal file
//        NULL);                    //  No attr. template
//
//    if (InputFile == INVALID_HANDLE_VALUE) {
//        wprintf(L"Cannot open \t%s\n", zFileIn);
//        goto done;
//    }
//
//    //  Get input file size.
//    Success = GetFileSizeEx(InputFile, &FileSize);
//    if ((!Success)||(FileSize.QuadPart > 0xFFFFFFFF)) {
//        wprintf(L"Cannot get input file size or file is larger than 4GB.\n");       
//        goto done;
//    }
//    InputFileSize = FileSize.LowPart;
//
//    //  Allocate memory for file content.
//    InputBuffer = (PBYTE)malloc(InputFileSize);
//    if (!InputBuffer) {
//        wprintf(L"Cannot allocate memory for uncompressed buffer.\n");
//        goto done;
//    }                             
//
//    //  Read input file.
//    Success = ReadFile(InputFile, InputBuffer, InputFileSize, &ByteRead, NULL);
//    if ((!Success)||(ByteRead != InputFileSize)) {
//        wprintf(L"Cannot read from \t%s\n", zFileIn);
//        goto done;
//    }
//
//    //  Open an empty file for writing, if exist, overwrite it.
//    CompressedFile = CreateFile(
//        zFileOut,                 //  Compressed file name
//        GENERIC_WRITE|DELETE,     //  Open for writing; delete if cannot compress
//        0,                        //  Do not share
//        NULL,                     //  Default security
//        CREATE_ALWAYS,            //  Create a new file; if exist, overwrite it
//        FILE_ATTRIBUTE_NORMAL,    //  Normal file
//        NULL);                    //  No template
//
//    if (CompressedFile == INVALID_HANDLE_VALUE) {
//        wprintf(L"Cannot create file \t%s\n", zFileOut);
//        goto done;
//    }
//
//    //  Create an XpressHuff compressor.
//    Success = CreateCompressor(
//        COMPRESS_ALGORITHM_XPRESS_HUFF, //  Compression Algorithm
//        NULL,                           //  Optional allocation routine
//        &Compressor);                   //  Handle
//
//    if (!Success) {
//        wprintf(L"Cannot create a compressor %d.\n", GetLastError());
//        goto done;
//    }
//
//    //  Query compressed buffer size.
//    Success = Compress(
//        Compressor,                  //  Compressor Handle
//        InputBuffer,                 //  Input buffer, Uncompressed data
//        InputFileSize,               //  Uncompressed data size
//        NULL,                        //  Compressed Buffer
//        0,                           //  Compressed Buffer size
//        &CompressedBufferSize);      //  Compressed Data size
//
//    //  Allocate memory for compressed buffer.
//    if (!Success) {
//        DWORD ErrorCode = GetLastError();
//
//        if (ErrorCode != ERROR_INSUFFICIENT_BUFFER)
//        {
//            wprintf(L"Cannot compress data: %d.\n", ErrorCode);
//            goto done;
//        }
//           
//        CompressedBuffer = (PBYTE)malloc(CompressedBufferSize);
//        if (!CompressedBuffer) {
//            wprintf(L"Cannot allocate memory for compressed buffer.\n");
//            goto done;
//        }
//    }
//
//    StartTime = GetTickCount64();
//
//    //  Call Compress() again to do real compression and output the compressed
//    //  data to CompressedBuffer.
//    Success = Compress(
//        Compressor,             //  Compressor Handle
//        InputBuffer,            //  Input buffer, Uncompressed data
//        InputFileSize,          //  Uncompressed data size
//        CompressedBuffer,       //  Compressed Buffer
//        CompressedBufferSize,   //  Compressed Buffer size
//        &CompressedDataSize);   //  Compressed Data size
//
//    if (!Success) {
//        wprintf(L"Cannot compress data: %d\n", GetLastError());
//        goto done;
//    }
//
//    EndTime = GetTickCount64();
//
//    //  Get compression time.
//    TimeDuration = (EndTime - StartTime) / 1000.0;
//
//    //  Write compressed data to output file.
//    Success = WriteFile(
//        CompressedFile,     //  File handle
//        CompressedBuffer,   //  Start of data to write
//        (DWORD)CompressedDataSize, //  Number of byte to write
//        &ByteWritten,       //  Number of byte written
//        NULL);              //  No overlapping structure
//
//    if ((ByteWritten != CompressedDataSize) || (!Success)) {
//        wprintf(L"Cannot write compressed data to file: %d.\n", GetLastError());
//        goto done;
//    }
//
//    wprintf(
//        L"Input file size: %d; Compressed Size: %d\n",
//        InputFileSize,
//        (DWORD)CompressedDataSize);
//    wprintf(L"Compression Time(Exclude I/O): %.2f seconds\n", TimeDuration);
//    wprintf(L"File Compressed.\n");
//
//    DeleteTargetFile = FALSE;
//
//done:
//    if (Compressor != NULL) {
//        CloseCompressor(Compressor);
//    }
//
//    if (CompressedBuffer) {
//        free(CompressedBuffer);
//    }
//
//    if (InputBuffer) {
//        free(InputBuffer);
//    }
//
//    if (InputFile != INVALID_HANDLE_VALUE) {
//        CloseHandle(InputFile);
//    }
//
//    if (CompressedFile != INVALID_HANDLE_VALUE) {
//        //  Compression fails, delete the compressed file.
//        if (DeleteTargetFile)
//        {
//            FILE_DISPOSITION_INFO fdi;
//            fdi.DeleteFile = TRUE;      //  Marking for deletion
//            Success = SetFileInformationByHandle(
//                CompressedFile,
//                FileDispositionInfo,
//                &fdi,
//                sizeof(FILE_DISPOSITION_INFO));   
//            if (!Success) {
//                wprintf(L"Cannot delete corrupted compressed file.\n");
//            }
//        }
//        CloseHandle(CompressedFile);
//    }
//}

// NTDLL version
// This version supports either COMPRESSION_FORMAT_LZNT1 (faster) or COMPRESSION_FORMAT_XPRESS (smaller size)
//
void wmain(_In_ int argc, _In_ WCHAR *argv[]) {
    COMPRESSOR_HANDLE Compressor    = NULL;
    PBYTE CompressedBuffer          = NULL;
    PBYTE InputBuffer               = NULL;
    HANDLE InputFile                = INVALID_HANDLE_VALUE;
    HANDLE CompressedFile           = INVALID_HANDLE_VALUE;   
    BOOL DeleteTargetFile           = TRUE;
    BOOL Success;
    DWORD InputFileSize, ByteRead, ByteWritten;
    LARGE_INTEGER FileSize;   
    ULONGLONG StartTime, EndTime;
    double TimeDuration;

    if (argc != 2) {
        wprintf(L"Usage:\n\t%s <input_file_name>\n", argv[0]);
        return;
    }

    WCHAR zFileIn[MAX_PATH] = { 0 }; wcscopy(zFileIn, argv[1]);
    WCHAR zFileOut[MAX_PATH] = { 0 };
    WCHAR drive[_MAX_DRIVE], dir[_MAX_DIR], fname[_MAX_FNAME], ext[_MAX_EXT];
    wsplitpath(zFileIn, drive, dir, fname, ext);

    wcscopy(ext, L".zor");
    wcscopy(zFileOut, drive); Add_Str(zFileOut, dir); Add_Str(zFileOut, fname); Add_Str(zFileOut, ext);

    //  Open input file for reading, existing file only.
    InputFile = CreateFile(zFileIn, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
    if (InputFile == INVALID_HANDLE_VALUE) {
        wprintf(L"Cannot open \t%s\n", zFileIn);
        goto done;
    }

    //  Get input file size.
    Success = GetFileSizeEx(InputFile, &FileSize);
    if ((!Success)||(FileSize.QuadPart > 0xFFFFFFFF)) {
        wprintf(L"Cannot get input file size or file is larger than 4GB.\n");       
        goto done;
    }

    InputFileSize = FileSize.LowPart;

    //  Allocate memory for file content.
    InputBuffer = (PBYTE)malloc(InputFileSize);
    if (!InputBuffer) {
        wprintf(L"Cannot allocate memory for uncompressed buffer.\n");
        goto done;
    } 

    //  Read input file.
    Success = ReadFile(InputFile, InputBuffer, InputFileSize, &ByteRead, NULL);
    if ((!Success) || (ByteRead != InputFileSize)) {
        wprintf(L"Cannot read from \t%s\n", zFileIn);
        goto done;
    }

    //  Open an empty file for writing, if exist, overwrite it.
    CompressedFile = CreateFile(zFileOut, GENERIC_WRITE|DELETE, 0, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL);
    if (CompressedFile == INVALID_HANDLE_VALUE) {
        wprintf(L"Cannot create file \t%s\n", zFileOut);
        goto done;
    }

    DWORD nSizeCBUF = 0, nSizeWorkSpace = 0;
    //RtlGetCompressionWorkSpaceSize(COMPRESSION_FORMAT_LZNT1, nSizeCBUF, nSizeWorkSpace);
    RtlGetCompressionWorkSpaceSize(COMPRESSION_FORMAT_XPRESS, nSizeCBUF, nSizeWorkSpace);
    char* szWorkBuffer =  new char[nSizeCBUF + 1]; memset(szWorkBuffer, 0, nSizeCBUF + 1);
    char* szCBUF = new char[InputFileSize + 1]; memset(szCBUF, 0, InputFileSize + 1);

    StartTime = GetTickCount64();

    DWORD FinalSize = 0;
    long nRet = RtlCompressBuffer (COMPRESSION_FORMAT_XPRESS, (PUCHAR) InputBuffer, InputFileSize, (PUCHAR) szCBUF, InputFileSize, 4096, FinalSize, szWorkBuffer);
    if (nRet == 0) { //  Write compressed data to output file.
        Success = WriteFile(CompressedFile,     //  File handle
                            szCBUF,             //  Start of data to write
                            FinalSize,          //  Number of byte to write
                            &ByteWritten,       //  Number of byte written
                            NULL);              //  No overlapping structure
    }

    delete [] szWorkBuffer;
    delete [] szCBUF;

    if (nRet) {
        wprintf(L"Cannot compress data: %d\n", GetLastError());
        goto done;
    }

    EndTime = GetTickCount64();

    //  Get compression time.
    TimeDuration = (EndTime - StartTime) / 1000.0;

    if ((ByteWritten != FinalSize) || (!Success)) {
        wprintf(L"Cannot write compressed data to file: %d.\n", GetLastError());
        goto done;
    }

    wprintf(L"Input file size: %d; Compressed Size: %d\n", InputFileSize, (DWORD)FinalSize);
    wprintf(L"Compression Time(Exclude I/O): %.2f seconds\n", TimeDuration);
    wprintf(L"File Compressed.\n");

    DeleteTargetFile = FALSE;

done:
    if (InputFile != INVALID_HANDLE_VALUE) {
        CloseHandle(InputFile);
    }

    if (CompressedFile != INVALID_HANDLE_VALUE) {
        //  Compression fails, delete the compressed file.
        if (DeleteTargetFile)  {
            FILE_DISPOSITION_INFO fdi;
            fdi.DeleteFile = TRUE;      //  Marking for deletion
            Success = SetFileInformationByHandle(
                CompressedFile,
                FileDispositionInfo,
                &fdi,
                sizeof(FILE_DISPOSITION_INFO));   
            if (!Success) {
                wprintf(L"Cannot delete corrupted compressed file.\n");
            }
        }
        CloseHandle(CompressedFile);
    }
}


[Michael Lobko-Lobanovsky]

Thanks for your effort to help, Patrice!

But please be patient: the Linuxoid code is heavily macro'ed  and written to an alien C standard. Yet I think I'll crack it to be usable in MS VC for our purposes.

(BTW what's the point in adding a compressor routine to your compressor.h if the only thing ObjReader would ever need is a de-compressor?)

[Patrice Terrier]

The compressor is only needed as an utility, to write new topics or edit an existing one with WordPad.

I have completed the pure ntdll version that would just add a few bytes to the OR code.
And the compression ratio is 2150400 / 245760 = 8.75 that is already a very good value using COMPRESSION_FORMAT_XPRESS.
The compression ratio for COMPRESSION_FORMAT_XPRESS_HUFF is 11.66, but that will produce larger OR code and requires extra dependencies at load time.


If the linux code is too much extra work (we need both compressor/decompressor), i would say that i could live with the 8.75 ratio that is the same on Seven and 10.