Author Topic: Early WIP on v2.55  (Read 132780 times)

Michael Lobko-Lobanovsky

  • Administrator
  • *****
  • Posts: 1481
Re: Early WIP on v2.55
« Reply #465 on: December 28, 2018, 05:26:20 pm »
... (the same to what i am doing in WinLIFT when skinning ListView).

BTW I'm using SendMessage(WM_SETREDRAW FALSE/TRUE) for both ListViews when populating them at model load time. Otherwise their scrollbars flicker like hell unskinned in the process. ;)

The RTF PowerBASIC project i sent you, was designed to work on Seven (written in 2016 by José Roca), thus i do not understand why it doesn't work by you  :-[

It does work for me under W7 but exactly the way any other genuine WordPad pages work: with indexed colors and image dither. If an ARGB PNG is embedded in the page markup manually, then its background (substrate) color is preserved but the foreground colors still get dithered and the image sizes on the page get broken to invisibility.

Evidently José Roca has no authority on my workstation despite all his numerous services to the BASIC Vaterland;D
Mike
(3.6GHz Intel Core i5 Quad w/ 16GB RAM, nVidia GTX 1060Ti w/ 6GB VRAM, Windows 7 Ultimate Sp1)

Patrice Terrier

  • Administrator
  • *****
  • Posts: 1980
    • zapsolution
Re: Early WIP on v2.55
« Reply #466 on: December 29, 2018, 09:32:17 pm »
I have almost completed my new Scrollbar.h to skin any control scrollbar on the fly.
That was a tedious work because i had to remember what i have done in WinLIFT, to add new support for the RICHEDIT control.
I have exported several WinLIFT internal functions to retrieve the bitmap handle of the scrollbar bitmap components.
Patrice
(Always working with the latest Windows version available...)

Michael Lobko-Lobanovsky

  • Administrator
  • *****
  • Posts: 1481
Re: Early WIP on v2.55
« Reply #467 on: December 29, 2018, 11:30:40 pm »
Amazing!

IIRC this is something I can't do yet. :)
Mike
(3.6GHz Intel Core i5 Quad w/ 16GB RAM, nVidia GTX 1060Ti w/ 6GB VRAM, Windows 7 Ultimate Sp1)

Patrice Terrier

  • Administrator
  • *****
  • Posts: 1980
    • zapsolution
Why i am using the current OS
« Reply #468 on: December 30, 2018, 05:23:24 pm »
Here is an example of the reason why i am always using the latest API and the current OS

The compress API is not available on Seven
https://docs.microsoft.com/en-us/windows/desktop/cmpapi/-compression-portal
this is what i wanted to use with my new help system to decrease the size of the .rtf files, and produce a lossless .ztf file that is almost half the size of the original when using embedded images.

Here is the link to the C++ compression/decompression examples to test in W10 console mode.
https://docs.microsoft.com/en-us/windows/desktop/cmpapi/using-the-compression-api-in-buffer-mode
and it is very fast…

Added
12-31-2018
I did several minor changes in OR, that means you will have to sync with me, once you get back from holidays. ;)
I have completed the W10 help support, i am very pleased of the result, and the new skinned scrollbar API works like a charm with it. The rtf compression is not added yet, but it is top on my list, once i have completed the writing of the basic documentation.

01-01-2018
I have fixed an array bug in ResetCamera.
I have written the missing MENU_FILE_REOPEN option.
I have completed the basic help documentation.
I have worked on a new model ;)

01-02-2019
And the whole combination of the new .ztf help files is only 808 Kb with all the embedded images!
Thanks to the W10 COMPRESS_ALGORITHM_XPRESS_HUFF that is amazing both in size and speed.
I have written two console utilities to compress/decompress the help files for the purpose of text edition (zCompress.exe and zDecompress.exe)
The attachment shows the new OR help skinned window on my W10 system (with mouse wheel support), using the new ScrollbarEx API control.

To compress a RTF file with zCompress, use this syntax from a prompt command line:
zCompress text.rtf (create automatically text.ztf into the same folder)

To decompress a ZTF file with zDeCompress, use this syntax from a prompt command line:
zDeCompress text.ztf (create automatically text.rtf into the same folder)

01-03-2019
Sync patch attached to this post…
« Last Edit: January 03, 2019, 10:58:06 am by Patrice Terrier »
Patrice
(Always working with the latest Windows version available...)

Patrice Terrier

  • Administrator
  • *****
  • Posts: 1980
    • zapsolution
Re: Early WIP on v2.55
« Reply #469 on: January 04, 2019, 10:28:41 am »
My friend,

To save your time, here is the uncompressed version of the rtf files, i forgot to put them in the previous patch.
They shouldn't be hard to convert to html, to let them work with HH on Seven.

I have also attached WinLIFT64 compiled with VS2010 for smaller binary size.
« Last Edit: January 04, 2019, 10:50:30 am by Patrice Terrier »
Patrice
(Always working with the latest Windows version available...)

Michael Lobko-Lobanovsky

  • Administrator
  • *****
  • Posts: 1481
Re: Early WIP on v2.55
« Reply #470 on: January 04, 2019, 12:57:10 pm »
Howdy Patrice,

You did an impressive amount of work these last few days! :)

Thanks for the uncompressed files -- with your permission, I'll use them to generate an equivalent .CHM file for the HH help. I'd also like to have a precompiled ObjReader64.exe to be able to test your help system at all because I can't even compile my own one as it requires a W10 version of Windows SDK -- something that I'm not going to touch until there are still pre-W10 OS'es that Microsoft keeps supporting.

Tell you more, though W8/8.1 can deal with the MS compression API in principle, it still requires an extra non-system Cabinet.dll library to be installed on the user PC.

OTOH I found some open-source Linuxoid code pieces that, apart from other functionality, are claimed to be able to decompress MS XPRESS_HUFF data, and they even claim that it works twice faster than the original. All in all, the code pulls in a lot of Lin/Win dependencies and compiles only with GCC. But I think I can isolate the decompression code, strip it of 3rd party references, make it MS VC friendly, and supply it with a standard MS (de)compression interface to be used literally in place of your W10-only APIs. Then we could have a reasonably small decompression include file to be linked statically in OR to make your system user-friendly on the pre-W10 platforms.

What's your opinion on this possibility?
Mike
(3.6GHz Intel Core i5 Quad w/ 16GB RAM, nVidia GTX 1060Ti w/ 6GB VRAM, Windows 7 Ultimate Sp1)

Patrice Terrier

  • Administrator
  • *****
  • Posts: 1980
    • zapsolution
Re: Early WIP on v2.55
« Reply #471 on: January 04, 2019, 02:32:09 pm »
Quote
Thanks for the uncompressed files -- with your permission, I'll use them to generate an equivalent .CHM file for the HH help.
Of course you have my permission, this is the reason why i sent you the uncompressed version.

I have attached the W10 ObjReader64 version compiled with VS2017, to let you try the ztf files in context.

Many of my latest projects (a.k.a. MBox64  and consor) are targeted to work with the latest API, and as much i disliked Windows 8, as much i am pleased of W10, that works very well even on my small ATOM Z8700 Surface (2 Gb RAM) that starts in less than 5 seconds with it!

I plan to test soon VS2019 to check the latest facilities provided by the new environment.

Quote
make it MS VC friendly, and supply it with a standard MS (de)compression interface to be used literally in place of your W10-only APIs
There is another API that could be used for compression/decompression, and it has been used by WinLIFT for years (see skSkiToDib).
And here is a link to the official documentation:
https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/content/ntifs/nf-ntifs-rtldecompressbuffer
Note: RtlCompressBuffer seems also to support COMPRESSION_FORMAT_XPRESS_HUFF.

Here is the complete WinLIFT encapsulation, that i could export and it should work on everything ;)
At the time i started to use it, it was a NTAPI undocumented couple of functions, and my code keep linking directly to the core ntdll API, bypassing all other extra encapsulations that have been added since XP.
Code: [Select]
long RtlDecompressBuffer (IN WORD nFormat, OUT CHAR* &szUBUF, IN DWORD nSizeUBUF, IN CHAR* szCBUF, IN DWORD nSizeCBUF, OUT DWORD &FinalSize) {
    long nRet = 0;
    HMODULE hDll = LoadLibrary(L"ntdll.dll");
    if (hDll) {
        long_proc (WORD, CHAR*, DWORD, CHAR*, DWORD, DWORD*);
        zProc hProc = (zProc) GetProcAddress(hDll, "RtlDecompressBuffer");
        if (hProc) { nRet = hProc(nFormat, szUBUF, nSizeUBUF, szCBUF, nSizeCBUF, &FinalSize); }
    }
    return nRet;
}

long RtlGetCompressionWorkSpaceSize (IN USHORT nFormat, OUT DWORD &nSizeCBUF, OUT DWORD &nSizeWorkSpace) {
    long nRet = 0;
    HMODULE hDll = LoadLibrary(L"ntdll.dll");
    if (hDll) {
        long_proc (WORD, DWORD*, DWORD*);
        zProc hProc = (zProc) GetProcAddress(hDll, "RtlGetCompressionWorkSpaceSize");
        if (hProc) { nRet = hProc(nFormat, &nSizeCBUF, &nSizeWorkSpace); }
    }
    return nRet;
}

long RtlCompressBuffer (IN WORD nFormat, IN PUCHAR szUBUF, IN DWORD nSizeUBUF, OUT PUCHAR szCBUF, IN DWORD nSizeCBUF, IN DWORD nSizeChunk,
                        OUT DWORD &FinalSize, IN PVOID WorkSpace) {
    long nRet = 0;
    HMODULE hDll = LoadLibrary(L"ntdll.dll");
    if (hDll) {
        long_proc (WORD, PUCHAR, DWORD, PUCHAR, DWORD, DWORD, DWORD*, PVOID);
        zProc hProc = (zProc) GetProcAddress(hDll, "RtlCompressBuffer");
        if (hProc) { nRet = hProc(nFormat, szUBUF, nSizeUBUF, szCBUF, nSizeCBUF, nSizeChunk, &FinalSize, WorkSpace); }
    }
    return nRet;
}

HBITMAP skSkiToDib (IN WCHAR* szFile, OUT long &nReserved) { // dllexport
    HBITMAP nFunction = 0;
    HDC hIC = 0;
    HBITMAP hDIB = 0;
    if (szFile) {
        if (FileExist(szFile)) {
            HANDLE hFile = 0;
            if (zFOpen(szFile, 0, 2, hFile) == 0) {
                DWORD LenBufIn = zFlof(hFile);
                string sBuffer; sBuffer.assign(LenBufIn, 0);
                zFGet(hFile, sBuffer);
                zFClose(hFile);
                if (LenBufIn > 8) {
                    string sHeader = left$(sBuffer, 8);
                    string sMem;
                    short imgW = 0; sMem = mid$(sHeader, 1, 2); MoveMemory(&imgW, (CHAR*) sMem.c_str(), 2);
                    short imgH = 0; sMem = mid$(sHeader, 3, 2); MoveMemory(&imgH, (CHAR*) sMem.c_str(), 2);
                    nReserved = 0; sMem = mid$(sHeader, 5, 4); MoveMemory(&nReserved, (CHAR*) sMem.c_str(), 4);
       
                    if (g_nUnlocked) { // Detect if (skin are valid
                        char zC[] = {(CHAR) nReserved};
                        g_nUnlocked = instr(0, g_sSkinKey, zC);
                    }
       
                    hIC = skDisplayDC();
                    hDIB = skCreateDIBSection(hIC, imgW, imgH, 32);
                    if (hDIB) {
                        BITMAP bm; GetObject(hDIB, sizeof(bm), &bm);
       
                        LenBufIn -= 8;
                        sBuffer = mid$(sBuffer, 9, LenBufIn);
       
                        // Very important to use the good size for the sBufOut buffer
                        // or we may have a STATUS_BAD_COMPRESSION_BUFFER error!
                        DWORD LenBufOut = bm.bmWidthBytes * bm.bmHeight;
                        //string sBufOut = sBufOut.assign(LenBufOut, 0);
                        char* szBufOut = new char[LenBufOut];
                        memset(szBufOut, 0, LenBufOut);
                        DWORD cBuffSize = 0;
                        long nRet = RtlDecompressBuffer (COMPRESSION_FORMAT_LZNT1, szBufOut, LenBufOut, (CHAR*) sBuffer.c_str(), LenBufIn, cBuffSize);
                        MoveMemory(bm.bmBits, szBufOut, min(LenBufOut, cBuffSize));
                        delete [] szBufOut;
                        if (nRet == 0) { nFunction = hDIB; }
                    }
                    DeleteDC(hIC);
                }
            }
        }
    }
    return nFunction;
}


PS: Tell me if you need the Cabinet.dll
« Last Edit: January 04, 2019, 03:57:26 pm by Patrice Terrier »
Patrice
(Always working with the latest Windows version available...)

Michael Lobko-Lobanovsky

  • Administrator
  • *****
  • Posts: 1481
Re: Early WIP on v2.55
« Reply #472 on: January 04, 2019, 04:25:48 pm »
Thanks for the executable!

Quote
Here is the complete WinLIFT encapsulation, that i could export and it should work on everything ;)

Would be useless as-is in this case because the decompressor that we need to re-create directly in ObjReader based on those exports would not support COMPRESSION_FORMAT_XPRESS_HUFF below MS Windows 8, and that's the format you're using in your ZTF help files. You'd have to also change your ZTF format to straight COMPRESSION_FORMAT_XPRESS to be compatible not only with W8/8.1/10 but also W7.

(How can you be so cold blooded and kill your app's compatibility with the potential sub-W10 user base with your own hands anyway? :-\)

Quote
Tell me if you need the Cabinet.dll

No thanks, I'm not planning on testing OR under W8/8.1 at this point in time.
Mike
(3.6GHz Intel Core i5 Quad w/ 16GB RAM, nVidia GTX 1060Ti w/ 6GB VRAM, Windows 7 Ultimate Sp1)

Patrice Terrier

  • Administrator
  • *****
  • Posts: 1980
    • zapsolution
Re: Early WIP on v2.55
« Reply #473 on: January 04, 2019, 05:49:57 pm »
Quote
How can you be so cold blooded and kill your app's compatibility with the potential sub-W10 user base with your own hands anyway?
Because here in France most peoples have moved fast to W10, and the private users even faster than the enterprises.
All my friends around me, have moved on, they were reluctant with 8 and 9 (8.1), but 10 has got all the suffrages.
See what ZDnet says about the french market
https://www.zdnet.fr/actualites/chiffres-cles-les-systemes-d-exploitation-sur-pc-39790131.htm

The only other large OS base, used here, is Android and i just bought on Amazon a new 8.4 HUAWEI Mediapad 5, that is a true jewel with its 2K screen.
https://consumer.huawei.com/fr/tablets/mediapad-m5/

IOS is still used by the bobos and the geeks, but the real big market here is W10.

I had the same problem at the time of VISTA, because i moved fast to DWM, while most people were still using XP, but that give me a big boost on my competitors because i learned how to use the new API(s) before all of them, that helped me to make a success of GDImage in the WinDev community.

Quote
Would be useless as-is in this case because the decompressor that we need to re-create directly in ObjReader based on those exports would not support COMPRESSION_FORMAT_XPRESS_HUFF below MS Windows 8,
I could use the same compression that is used in my .ski or my Fly audio files, that is working on XP+, or i could just use the rtf uncompressed as well ;)
I shall make a size comparison of the two compression systems and will let you know the result.
« Last Edit: January 04, 2019, 05:53:26 pm by Patrice Terrier »
Patrice
(Always working with the latest Windows version available...)

Michael Lobko-Lobanovsky

  • Administrator
  • *****
  • Posts: 1481
Re: Early WIP on v2.55
« Reply #474 on: January 04, 2019, 06:28:11 pm »
Quote
https://www.zdnet.fr/actualites/chiffres-cles-les-systemes-d-exploitation-sur-pc-39790131.htm

But you can see yourself that every 3rd person in France is still using Windows 7! ??? IMHO it is unwise to write them all off your list of potential users...
Mike
(3.6GHz Intel Core i5 Quad w/ 16GB RAM, nVidia GTX 1060Ti w/ 6GB VRAM, Windows 7 Ultimate Sp1)

Patrice Terrier

  • Administrator
  • *****
  • Posts: 1980
    • zapsolution
Re: Early WIP on v2.55
« Reply #475 on: January 04, 2019, 08:02:46 pm »
I wrote a quick compressor project based on NTDLL for comparison purpose

Uncompressed file: 006Lighting.rtf 2150400 bytes

and the result are

with the W10 Cabinet.dll using COMPRESSION_FORMAT_XPRESS_HUFF the file is 184320 bytes

with NTDLL using COMPRESSION_FORMAT_XPRESS the file is 245760 bytes

with NTDLL using COMPRESSION_FORMAT_LZNT1 the file is 425984 bytes (smallest compression ratio, however the fastest)
Patrice
(Always working with the latest Windows version available...)

Michael Lobko-Lobanovsky

  • Administrator
  • *****
  • Posts: 1481
Re: Early WIP on v2.55
« Reply #476 on: January 04, 2019, 10:08:40 pm »
Please let your message stay intact for comparison purposes (do not delete it). I'll see if I can do anything with the Linux code and COMPRESSION_FORMAT_XPRESS_HUFF decompression tomorrow.
Mike
(3.6GHz Intel Core i5 Quad w/ 16GB RAM, nVidia GTX 1060Ti w/ 6GB VRAM, Windows 7 Ultimate Sp1)

Patrice Terrier

  • Administrator
  • *****
  • Posts: 1980
    • zapsolution
Re: Early WIP on v2.55
« Reply #477 on: January 04, 2019, 10:44:34 pm »
Just in case, here is the console code i wrote to compress the test file.
As you can see i could easily switch from one to another to select one that give the best result in the old and new Windows paradigm ;)

Tools.h
Code: [Select]
#pragma once
using namespace std;

#define void_proc_c typedef void (__cdecl *zProc)
#define long_proc_c typedef long (__cdecl *zProc)
#define long_proc typedef long (__stdcall *zProc)

WCHAR* wcscopy (IN WCHAR* dst, IN const WCHAR* src) {
    // Warning, there is no check for buffer overflow !
    long nLen = lstrlen(src);
    //for (long K = 0; K < nLen; K++) {
    //    dst[K] = src[K];
    //}
    MoveMemory(dst, src, nLen * sizeof(WCHAR));
    dst[nLen] = L'\0';
    return dst;
}

HMODULE MSVCRT() {
    static HMODULE hModule;
    if (hModule == 0) { hModule = LoadLibrary(L"MSVCRT"); }
    return hModule;
}

void wsplitpath(IN WCHAR* FullPath, OUT WCHAR* Drive, OUT WCHAR* Dir, OUT WCHAR* FileName, OUT WCHAR* Ext) {
    HMODULE hModule = MSVCRT();
    if (hModule) {
        void_proc_c(WCHAR*, WCHAR*, WCHAR*, WCHAR*, WCHAR*);
        zProc hProc = (zProc)GetProcAddress(hModule, "_wsplitpath");
        if (hProc) { hProc(FullPath, Drive, Dir, FileName, Ext); }
    }
}

long Add_Str(OUT WCHAR* dest, IN WCHAR* srce) {
    long nRet = -1; // Error
    HMODULE hModule =  MSVCRT();
    if (hModule) {
        long_proc_c (WCHAR*, WCHAR*, size_t);
        zProc hProc = (zProc)GetProcAddress(hModule, "wcsncat");
        if (hProc) { nRet = hProc(dest, srce, _TRUNCATE); }
    }
    return nRet;
}

HMODULE NTDLL() {
    static HMODULE hModule;
    if (hModule == 0) { hModule = LoadLibrary(L"ntdll"); }
    return hModule;
}

long RtlDecompressBuffer (IN WORD nFormat, OUT CHAR* &szUBUF, IN DWORD nSizeUBUF, IN CHAR* szCBUF, IN DWORD nSizeCBUF, OUT DWORD &FinalSize) {
    long nRet = 0;
    HMODULE hModule =  NTDLL();
    if (hModule) {
        long_proc (WORD, CHAR*, DWORD, CHAR*, DWORD, DWORD*);
        zProc hProc = (zProc) GetProcAddress(hModule, "RtlDecompressBuffer");
        if (hProc) { nRet = hProc(nFormat, szUBUF, nSizeUBUF, szCBUF, nSizeCBUF, &FinalSize); }
    }
    return nRet;
}

long RtlGetCompressionWorkSpaceSize (IN USHORT nFormat, OUT DWORD &nSizeCBUF, OUT DWORD &nSizeWorkSpace) {
    long nRet = 0;
    HMODULE hModule =  NTDLL();
    if (hModule) {
        long_proc (WORD, DWORD*, DWORD*);
        zProc hProc = (zProc) GetProcAddress(hModule, "RtlGetCompressionWorkSpaceSize");
        if (hProc) { nRet = hProc(nFormat, &nSizeCBUF, &nSizeWorkSpace); }
    }
    return nRet;
}

long RtlCompressBuffer (IN WORD nFormat, IN PUCHAR szUBUF, IN DWORD nSizeUBUF, OUT PUCHAR szCBUF, IN DWORD nSizeCBUF, IN DWORD nSizeChunk,
                        OUT DWORD &FinalSize, IN PVOID WorkSpace) {
    long nRet = 0;
    HMODULE hModule =  NTDLL();
    if (hModule) {
        long_proc (WORD, PUCHAR, DWORD, PUCHAR, DWORD, DWORD, DWORD*, PVOID);
        zProc hProc = (zProc) GetProcAddress(hModule, "RtlCompressBuffer");
        if (hProc) { nRet = hProc(nFormat, szUBUF, nSizeUBUF, szCBUF, nSizeCBUF, nSizeChunk, &FinalSize, WorkSpace); }
    }
    return nRet;
}

and wmain
Code: [Select]
// zCompress.cpp : This file contains the 'main' function. Program execution begins and ends there.
#include <iostream>


// Run program: Ctrl + F5 or Debug > Start Without Debugging menu
// Debug program: F5 or Debug > Start Debugging menu

// Tips for Getting Started:
//   1. Use the Solution Explorer window to add/manage files
//   2. Use the Team Explorer window to connect to source control
//   3. Use the Output window to see build output and other messages
//   4. Use the Error List window to view errors
//   5. Go to Project > Add New Item to create new code files, or Project > Add Existing Item to add existing code files to the project
//   6. In the future, to open this project again, go to File > Open > Project and select the .sln file
#include <Windows.h>
#include <stdio.h>
#include <compressapi.h>
#include "Tools.h"

// Windows 10, Cabinet.dll version
// Warning, the COMPRESS_ALGORITHM_XPRESS_HUFF is not supported on older OS
//
//void wmain(_In_ int argc, _In_ WCHAR *argv[]) {
//    COMPRESSOR_HANDLE Compressor    = NULL;
//    PBYTE CompressedBuffer          = NULL;
//    PBYTE InputBuffer               = NULL;
//    HANDLE InputFile                = INVALID_HANDLE_VALUE;
//    HANDLE CompressedFile           = INVALID_HANDLE_VALUE;   
//    BOOL DeleteTargetFile           = TRUE;
//    BOOL Success;
//    SIZE_T CompressedDataSize, CompressedBufferSize;
//    DWORD InputFileSize, ByteRead, ByteWritten;
//    LARGE_INTEGER FileSize;   
//    ULONGLONG StartTime, EndTime;
//    double TimeDuration;
//
//    if (argc != 2) {
//        wprintf(L"Usage:\n\t%s <input_file_name>\n", argv[0]);
//        return;
//    }
//
//    WCHAR zFileIn[MAX_PATH] = { 0 }; wcscopy(zFileIn, argv[1]);
//    WCHAR zFileOut[MAX_PATH] = { 0 };
//    WCHAR drive[_MAX_DRIVE], dir[_MAX_DIR], fname[_MAX_FNAME], ext[_MAX_EXT];
//    wsplitpath(zFileIn, drive, dir, fname, ext);
//    if (wcsstr(L".rtf", ext) == 0) {
//        wprintf(L"The input file must be a valid .rtf file.\n");
//        goto done;
//    }
//
//    wcscopy(ext, L".ztf");
//    wcscopy(zFileOut, drive); Add_Str(zFileOut, dir); Add_Str(zFileOut, fname); Add_Str(zFileOut, ext);
//
//    //  Open input file for reading, existing file only.
//    InputFile = CreateFile(
//        zFileIn,                  //  Input file name
//        GENERIC_READ,             //  Open for reading
//        FILE_SHARE_READ,          //  Share for read
//        NULL,                     //  Default security
//        OPEN_EXISTING,            //  Existing file only
//        FILE_ATTRIBUTE_NORMAL,    //  Normal file
//        NULL);                    //  No attr. template
//
//    if (InputFile == INVALID_HANDLE_VALUE) {
//        wprintf(L"Cannot open \t%s\n", zFileIn);
//        goto done;
//    }
//
//    //  Get input file size.
//    Success = GetFileSizeEx(InputFile, &FileSize);
//    if ((!Success)||(FileSize.QuadPart > 0xFFFFFFFF)) {
//        wprintf(L"Cannot get input file size or file is larger than 4GB.\n");       
//        goto done;
//    }
//    InputFileSize = FileSize.LowPart;
//
//    //  Allocate memory for file content.
//    InputBuffer = (PBYTE)malloc(InputFileSize);
//    if (!InputBuffer) {
//        wprintf(L"Cannot allocate memory for uncompressed buffer.\n");
//        goto done;
//    }                             
//
//    //  Read input file.
//    Success = ReadFile(InputFile, InputBuffer, InputFileSize, &ByteRead, NULL);
//    if ((!Success)||(ByteRead != InputFileSize)) {
//        wprintf(L"Cannot read from \t%s\n", zFileIn);
//        goto done;
//    }
//
//    //  Open an empty file for writing, if exist, overwrite it.
//    CompressedFile = CreateFile(
//        zFileOut,                 //  Compressed file name
//        GENERIC_WRITE|DELETE,     //  Open for writing; delete if cannot compress
//        0,                        //  Do not share
//        NULL,                     //  Default security
//        CREATE_ALWAYS,            //  Create a new file; if exist, overwrite it
//        FILE_ATTRIBUTE_NORMAL,    //  Normal file
//        NULL);                    //  No template
//
//    if (CompressedFile == INVALID_HANDLE_VALUE) {
//        wprintf(L"Cannot create file \t%s\n", zFileOut);
//        goto done;
//    }
//
//    //  Create an XpressHuff compressor.
//    Success = CreateCompressor(
//        COMPRESS_ALGORITHM_XPRESS_HUFF, //  Compression Algorithm
//        NULL,                           //  Optional allocation routine
//        &Compressor);                   //  Handle
//
//    if (!Success) {
//        wprintf(L"Cannot create a compressor %d.\n", GetLastError());
//        goto done;
//    }
//
//    //  Query compressed buffer size.
//    Success = Compress(
//        Compressor,                  //  Compressor Handle
//        InputBuffer,                 //  Input buffer, Uncompressed data
//        InputFileSize,               //  Uncompressed data size
//        NULL,                        //  Compressed Buffer
//        0,                           //  Compressed Buffer size
//        &CompressedBufferSize);      //  Compressed Data size
//
//    //  Allocate memory for compressed buffer.
//    if (!Success) {
//        DWORD ErrorCode = GetLastError();
//
//        if (ErrorCode != ERROR_INSUFFICIENT_BUFFER)
//        {
//            wprintf(L"Cannot compress data: %d.\n", ErrorCode);
//            goto done;
//        }
//           
//        CompressedBuffer = (PBYTE)malloc(CompressedBufferSize);
//        if (!CompressedBuffer) {
//            wprintf(L"Cannot allocate memory for compressed buffer.\n");
//            goto done;
//        }
//    }
//
//    StartTime = GetTickCount64();
//
//    //  Call Compress() again to do real compression and output the compressed
//    //  data to CompressedBuffer.
//    Success = Compress(
//        Compressor,             //  Compressor Handle
//        InputBuffer,            //  Input buffer, Uncompressed data
//        InputFileSize,          //  Uncompressed data size
//        CompressedBuffer,       //  Compressed Buffer
//        CompressedBufferSize,   //  Compressed Buffer size
//        &CompressedDataSize);   //  Compressed Data size
//
//    if (!Success) {
//        wprintf(L"Cannot compress data: %d\n", GetLastError());
//        goto done;
//    }
//
//    EndTime = GetTickCount64();
//
//    //  Get compression time.
//    TimeDuration = (EndTime - StartTime) / 1000.0;
//
//    //  Write compressed data to output file.
//    Success = WriteFile(
//        CompressedFile,     //  File handle
//        CompressedBuffer,   //  Start of data to write
//        (DWORD)CompressedDataSize, //  Number of byte to write
//        &ByteWritten,       //  Number of byte written
//        NULL);              //  No overlapping structure
//
//    if ((ByteWritten != CompressedDataSize) || (!Success)) {
//        wprintf(L"Cannot write compressed data to file: %d.\n", GetLastError());
//        goto done;
//    }
//
//    wprintf(
//        L"Input file size: %d; Compressed Size: %d\n",
//        InputFileSize,
//        (DWORD)CompressedDataSize);
//    wprintf(L"Compression Time(Exclude I/O): %.2f seconds\n", TimeDuration);
//    wprintf(L"File Compressed.\n");
//
//    DeleteTargetFile = FALSE;
//
//done:
//    if (Compressor != NULL) {
//        CloseCompressor(Compressor);
//    }
//
//    if (CompressedBuffer) {
//        free(CompressedBuffer);
//    }
//
//    if (InputBuffer) {
//        free(InputBuffer);
//    }
//
//    if (InputFile != INVALID_HANDLE_VALUE) {
//        CloseHandle(InputFile);
//    }
//
//    if (CompressedFile != INVALID_HANDLE_VALUE) {
//        //  Compression fails, delete the compressed file.
//        if (DeleteTargetFile)
//        {
//            FILE_DISPOSITION_INFO fdi;
//            fdi.DeleteFile = TRUE;      //  Marking for deletion
//            Success = SetFileInformationByHandle(
//                CompressedFile,
//                FileDispositionInfo,
//                &fdi,
//                sizeof(FILE_DISPOSITION_INFO));   
//            if (!Success) {
//                wprintf(L"Cannot delete corrupted compressed file.\n");
//            }
//        }
//        CloseHandle(CompressedFile);
//    }
//}

// NTDLL version
// This version supports either COMPRESSION_FORMAT_LZNT1 (faster) or COMPRESSION_FORMAT_XPRESS (smaller size)
//
void wmain(_In_ int argc, _In_ WCHAR *argv[]) {
    COMPRESSOR_HANDLE Compressor    = NULL;
    PBYTE CompressedBuffer          = NULL;
    PBYTE InputBuffer               = NULL;
    HANDLE InputFile                = INVALID_HANDLE_VALUE;
    HANDLE CompressedFile           = INVALID_HANDLE_VALUE;   
    BOOL DeleteTargetFile           = TRUE;
    BOOL Success;
    DWORD InputFileSize, ByteRead, ByteWritten;
    LARGE_INTEGER FileSize;   
    ULONGLONG StartTime, EndTime;
    double TimeDuration;

    if (argc != 2) {
        wprintf(L"Usage:\n\t%s <input_file_name>\n", argv[0]);
        return;
    }

    WCHAR zFileIn[MAX_PATH] = { 0 }; wcscopy(zFileIn, argv[1]);
    WCHAR zFileOut[MAX_PATH] = { 0 };
    WCHAR drive[_MAX_DRIVE], dir[_MAX_DIR], fname[_MAX_FNAME], ext[_MAX_EXT];
    wsplitpath(zFileIn, drive, dir, fname, ext);

    wcscopy(ext, L".zor");
    wcscopy(zFileOut, drive); Add_Str(zFileOut, dir); Add_Str(zFileOut, fname); Add_Str(zFileOut, ext);

    //  Open input file for reading, existing file only.
    InputFile = CreateFile(zFileIn, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
    if (InputFile == INVALID_HANDLE_VALUE) {
        wprintf(L"Cannot open \t%s\n", zFileIn);
        goto done;
    }

    //  Get input file size.
    Success = GetFileSizeEx(InputFile, &FileSize);
    if ((!Success)||(FileSize.QuadPart > 0xFFFFFFFF)) {
        wprintf(L"Cannot get input file size or file is larger than 4GB.\n");       
        goto done;
    }

    InputFileSize = FileSize.LowPart;

    //  Allocate memory for file content.
    InputBuffer = (PBYTE)malloc(InputFileSize);
    if (!InputBuffer) {
        wprintf(L"Cannot allocate memory for uncompressed buffer.\n");
        goto done;
    } 

    //  Read input file.
    Success = ReadFile(InputFile, InputBuffer, InputFileSize, &ByteRead, NULL);
    if ((!Success) || (ByteRead != InputFileSize)) {
        wprintf(L"Cannot read from \t%s\n", zFileIn);
        goto done;
    }

    //  Open an empty file for writing, if exist, overwrite it.
    CompressedFile = CreateFile(zFileOut, GENERIC_WRITE|DELETE, 0, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL);
    if (CompressedFile == INVALID_HANDLE_VALUE) {
        wprintf(L"Cannot create file \t%s\n", zFileOut);
        goto done;
    }

    DWORD nSizeCBUF = 0, nSizeWorkSpace = 0;
    //RtlGetCompressionWorkSpaceSize(COMPRESSION_FORMAT_LZNT1, nSizeCBUF, nSizeWorkSpace);
    RtlGetCompressionWorkSpaceSize(COMPRESSION_FORMAT_XPRESS, nSizeCBUF, nSizeWorkSpace);
    char* szWorkBuffer =  new char[nSizeCBUF + 1]; memset(szWorkBuffer, 0, nSizeCBUF + 1);
    char* szCBUF = new char[InputFileSize + 1]; memset(szCBUF, 0, InputFileSize + 1);

    StartTime = GetTickCount64();

    DWORD FinalSize = 0;
    long nRet = RtlCompressBuffer (COMPRESSION_FORMAT_XPRESS, (PUCHAR) InputBuffer, InputFileSize, (PUCHAR) szCBUF, InputFileSize, 4096, FinalSize, szWorkBuffer);
    if (nRet == 0) { //  Write compressed data to output file.
        Success = WriteFile(CompressedFile,     //  File handle
                            szCBUF,             //  Start of data to write
                            FinalSize,          //  Number of byte to write
                            &ByteWritten,       //  Number of byte written
                            NULL);              //  No overlapping structure
    }

    delete [] szWorkBuffer;
    delete [] szCBUF;

    if (nRet) {
        wprintf(L"Cannot compress data: %d\n", GetLastError());
        goto done;
    }

    EndTime = GetTickCount64();

    //  Get compression time.
    TimeDuration = (EndTime - StartTime) / 1000.0;

    if ((ByteWritten != FinalSize) || (!Success)) {
        wprintf(L"Cannot write compressed data to file: %d.\n", GetLastError());
        goto done;
    }

    wprintf(L"Input file size: %d; Compressed Size: %d\n", InputFileSize, (DWORD)FinalSize);
    wprintf(L"Compression Time(Exclude I/O): %.2f seconds\n", TimeDuration);
    wprintf(L"File Compressed.\n");

    DeleteTargetFile = FALSE;

done:
    if (InputFile != INVALID_HANDLE_VALUE) {
        CloseHandle(InputFile);
    }

    if (CompressedFile != INVALID_HANDLE_VALUE) {
        //  Compression fails, delete the compressed file.
        if (DeleteTargetFile)  {
            FILE_DISPOSITION_INFO fdi;
            fdi.DeleteFile = TRUE;      //  Marking for deletion
            Success = SetFileInformationByHandle(
                CompressedFile,
                FileDispositionInfo,
                &fdi,
                sizeof(FILE_DISPOSITION_INFO));   
            if (!Success) {
                wprintf(L"Cannot delete corrupted compressed file.\n");
            }
        }
        CloseHandle(CompressedFile);
    }
}
« Last Edit: January 04, 2019, 10:48:53 pm by Patrice Terrier »
Patrice
(Always working with the latest Windows version available...)

Michael Lobko-Lobanovsky

  • Administrator
  • *****
  • Posts: 1481
Re: Early WIP on v2.55
« Reply #478 on: January 05, 2019, 05:23:05 pm »
Thanks for your effort to help, Patrice!

But please be patient: the Linuxoid code is heavily macro'ed  and written to an alien C standard. Yet I think I'll crack it to be usable in MS VC for our purposes. ;)

(BTW what's the point in adding a compressor routine to your compressor.h if the only thing ObjReader would ever need is a de-compressor?)
Mike
(3.6GHz Intel Core i5 Quad w/ 16GB RAM, nVidia GTX 1060Ti w/ 6GB VRAM, Windows 7 Ultimate Sp1)

Patrice Terrier

  • Administrator
  • *****
  • Posts: 1980
    • zapsolution
Re: Early WIP on v2.55
« Reply #479 on: January 05, 2019, 06:04:57 pm »
The compressor is only needed as an utility, to write new topics or edit an existing one with WordPad.

I have completed the pure ntdll version that would just add a few bytes to the OR code.
And the compression ratio is 2150400 / 245760 = 8.75 that is already a very good value using COMPRESSION_FORMAT_XPRESS.
The compression ratio for COMPRESSION_FORMAT_XPRESS_HUFF is 11.66, but that will produce larger OR code and requires extra dependencies at load time.


If the linux code is too much extra work (we need both compressor/decompressor), i would say that i could live with the 8.75 ratio that is the same on Seven and 10.

« Last Edit: January 05, 2019, 06:07:44 pm by Patrice Terrier »
Patrice
(Always working with the latest Windows version available...)