Early WIP on v2.55

[Michael Lobko-Lobanovsky]

I have completed the pure ntdll version ...

OMG Patrice, that's yet another case when, as if competing who turns out to be the smartest of the two in the end, we're working on the same issue in parallel unawares of what the companion is/is not doing... What kept you so long before letting me know you've already decided what your personal preferences are as the project lead?

... that will produce larger OR code ...

That's correct. COMPRESSION_FORMAT_XPRESS_HUFF presupposes that, like any other modern deep lossless compression algo, the code uses extra Huffman hashing, which implies building and maintaining a dynamic dictionary whose physical size determines the final (de)compression ratio. All the maintenance code is implemented as __forceinline functions that are in fact macros expandable in-place throughout the code, which will certainly add much more binary code to the executable than just "a few bytes to the OR code" you developed. But that's a trade-off for an almost 50% deeper compression ratio than the raw XPRESS algo provides.

... and requires extra dependencies at load time.

But no, unlike the MS stuff you used to advocate, the Linuxoid code doesn't require any dependencies at either load time or afterwards. It is self-sufficient and functionally complete.

... (we need both compressor/decompressor) ...

No! We do not need a compressor in OR unless you aren't planning to edit its help files directly from its own environment. As things are, ATM your BOOL compressor(IN WCHAR* zFileIn, IN WCHAR* zFileOut) in compressor.h is pretty much dead code that bloats the OR binary because the project doesn't use the VC linker dead code removal option.

You've already implemented compression/decompression routines as standalone console utils that are unlikely to be used by anyone except you -- the author, creator and maintainer of ObjReader's RTF/ZTF help files. So why not keep them the way they are, based on the W10-only MS compression API?

_____________________________________________________

Anyway, if you "could live with the 8.75 ratio that is the same on Seven and 10", I hereby quit porting the Linuxoid code as a pointless, futile and wasteful affair leading knowingly to a dead end. Please send me your final NTDLL help compression code for sync'ing.

[Patrice Terrier]

Mike

There is clearly a misunderstood, the compressor is not intended to be inside of OR.
But i need an external compressor utility to convert the .rtf file or whatever to create the compressed file.

About my code, i already told you that compression based on the ntdll was part of WinLIFT, i just have to export the functions.

Also you told me that the rtf format i was using was not looking good on Seven, thus i thought that you wanted to stay with the HH solution that is chm based.

I have no problem using HH into OR.
And i am looking forward for your linuxoid  HUFF compression, that i will gladly use if it produces smaller and faster binary files.
I see no competition there, but a panel of choices for our finger tips.

As i told you before your holidays, i wanted for long to have my own custom help solution based on RTF, because i always found the creation of .chm files a tedious process, even with the help of Vizaac Helpmaker that i have used for years (they seem to have disappeared now from the net).

Adding compression to rtf, was part of my Help white paper, in order to keep the files small, and to avoid unsolicited edition of the provided documentation.
And to enforce the security i plan to use the same kind of header signature, borrowed to my WinLIFT's ".ski" files to protect the components inside of my commercial skin package collection.
This is also what i am doing with my fly audio files that are an OGG encrypted format, to be played only with FlyWorship.

[Patrice Terrier]

You've already implemented compression/decompression routines as standalone console utils that are unlikely to be used by anyone except you -- the author, creator and maintainer of ObjReader's RTF/ZTF help files. So why not keep them the way they are, based on the W10-only MS compression API?

Anyway, if you "could live with the 8.75 ratio that is the same on Seven and 10", I hereby quit porting the Linuxoid code as a pointless, futile and wasteful affair leading knowingly to a dead end. Please send me your final NTDLL help compression code for sync'ing.

To say the truth, having two distinct help file system this is what looks wasteful to me.

Thus, i cast my vote for the solution that would be the easiest to maintain for both of us.

If HH is needed for backward compatibility, then we can remove all the code from help.h, compress.h, and scrollbar.h, to reduce the code size (because i am using it only there).
And please send me the new chm file with the additions or corrections you may have done.

[Michael Lobko-Lobanovsky]

Patrice,

To say the truth, having two distinct help file system this is what looks wasteful to me.

We aren't having two distinct help systems. We offer conditional compilation that yields one single help system at the user's option if they want, and are able, to compile the app directly from the sources:

• either a W10-only ZTF-based system that is not backward compatible with either W7 (that's going to enjoy MS extended support for at least 1 calendar year to come) or W8.1 (that's going to enjoy MS extended support till at least the year 2023) and is thus useless for at least one third of ObjReader's potential user base both inside France and worldwide;
• or a universal CHM-based system that's both backward and forward compatible with all still existing Windows OS'es including W7, W8.1 and W10 and hopefully more if Bill Gates stays alive and kicking.

And now ask anyone on this forum which system they'd prefer to have in their precompiled ObjReader distros? This isn't a question of which (or whose) system is more compact byte-wise but rather which one is better suited for the variety of user setups that ObjReader is capable of running on. In all other respects, the both systems are nearly equal visually and functionally.

I haven't produced an equivalent CHM yet. I was busy reworking the code to incorporate a platform-independent XPRESS_HUFF decoder in your help system. And I made all the code fixes except one: the MS compression API supplies your ZTF files with a header of unknown format, and I can't yet make out exactly where the compressed data proper starts in a ZTF file. This is the last problem I'll be working on tomorrow. I can read the uncompressed buffer length from the ZTF file directly but the compressed buffer starting address will require some trial and error that's too late to rack my brains about tonight. I will compile the CHM file once I'm through with what I'm doing now -- it's trivial and not challenging at all.

[Patrice Terrier]

My friend,

I didn't wanted the help function to become a conditionnal compilation. I want just one same help system for everything, using the same basic files to produce the topic contents. Also we should use only one help GUI for ease of code maintenance, either aside of the slider that seems to have your preference or above the viewport as i have done.

So far, i wanted to go to full completion of what i had in mind, using Wordpad RTF files, compressed with the native API to produce a tiny help system. I succeed on this, and i am pleased with the work done that could be used in my other W10 projects. However i shall revert first to the ntdll compression, except if you succeed with your linux transcription that i would be glad to use with your permission.

In the case of our common OR project, it is very important that both of us could enjoy it with our respective OS of choice, my use of the cabinet.dll was an error because it is is not available on Seven.

[Patrice Terrier]

Ok, thank you very much for your effort!

I can live with the COMPRESSION_FORMAT_XPRESS ratio of 8.56 that is already very good (and fast).
This is what i will use now in new projects, previously i was using COMPRESSION_FORMAT_LZNT1 for compatibilty purpose with … XP, go figure

About the black background you get running the code on Seven, i have no idea of what could cause this, because i am using exactly the OLE code documented on MSDN here
https://docs.microsoft.com/en-us/windows/desktop/controls/using-rich-edit-com
perhaps this is another W10 specific API, because the documentation date is from May, 30, 2018.

There is an older example of 2005, on CodeProject, here
https://www.codeproject.com/Articles/9541/%2FArticles%2F9541%2FA-Rich-Edit-Control-That-Displays-Bitmaps-and-Othe
However the code style used by the author is too far from the SDK style i am familiar with, to let me use it.

[Patrice Terrier]

Mike

Here is the new XPRESS only compressor, based on ntdll.dll.

The Help folder, has both compressed and uncompressed file + 2 console utilities, to convert from one format to another.

[Patrice Terrier]

Oh No! 

with COMPRESSION_FORMAT_LZNT1, the ratio will be only 5.05
with COMPRESSION_FORMAT_LZNT1 | COMPRESSION_ENGINE_MAXIMUM, the ratio will be 5.73(but slower)

5 would be better than nothing 

[Michael Lobko-Lobanovsky]

... it's unlikely that something might distract me from my hobbies.

If you want to make the god laugh out loud in your face, just let him know about your intentions.

It's been a while since I checked ObjReader last on my 64-bit AMD/ATi W7 box. I did so today only to find that the ATi Radeon version of OpenGL refuses to compile the OR shaders. I had to remove the GL_EXT_gpu_shader4 extension that enabled us to render the model flat shaded in certain modes, and I had also to remove the initialization of shader uniforms in the GLSL code because it isn't supported in ATi's GLSL v1.2 that the shaders were using. nVidia seems to be much more permissive in this regard than its closest competitor.

Yet I found out that this PC is more tolerant to your .RTF help files. Of course 32-bit alpha transparent PNG's are out of the question as they get dithered as badly as on my main workstation, but what regards the sizes and shapes of the fonts, I can read practically any text that's there in the files.

So, it's an indication that the RichEdit distortions I'm seeing are likely to be a W7 video driver issue (both of my PCs have the latest drivers installed from their respective video card vendors), and no one knows how bad or good it may be on yet other people's PCs. I'll be trying to rework the .RTF's in my native W7 WordPad as planned until they look OK on both of my W7 machines.

[Patrice Terrier]

Same problems here with Intel HD Graphics 4000, i can't run OR anymore on my ASUS i5 Transformer. 

The solution could be perhaps to check the graphic card being used, and use these shaders only on nVidia ?

[Michael Lobko-Lobanovsky]

Here's the patch with the shaders and renderers fixed to run on my ATi Radeon. Hopefully it'll suit your 4000. Let me know if it works or not.

[Michael Lobko-Lobanovsky]

BTW do you mean to say your ASUS i5 Transformer runs exclusively on Intel HD Graphics 4000 HW and we can now create and test some specific code to add Intel recognition to our options? Can you send me a snapshot of your Transformer's Graphics info... message box if the patch I sent you works well?

[Patrice Terrier]

Yes, this ASUS Transformer has only on an Intel HD 4000, see the attached snapshot.

With your patch, everything seems to work, including the RTF help.
The only problem i have found so far, is when moving the slider, the dot bitmap is left behind and is moved only once i release the mouse button.

Of course there is no speed comparison with my nVidia ASUS Gamer computer.

[Michael Lobko-Lobanovsky]

Glad the patch worked for you!

Thanks for the Intel HD Graphics panel snapshot but I'd like to see ObjReader's message box as well, that pops up when clicking the video brand icon in the main window caption or choosing the Help->Graphics info... menu, if possible.

[Patrice Terrier]

Ok, here is the dialog.