Author Topic: TCLib - Replacement For MS C Runtime To Reduce Program Size  (Read 3802 times)

Frederick Harris

  • Newbie
  • *
  • Posts: 47
TCLib - Replacement For MS C Runtime To Reduce Program Size
« on: October 14, 2021, 09:02:41 pm »
     TCLib is a substitute C Runtime Library for Microsoft’s C Runtime Library.  It was inspired by Microsoft’s Matt Pietrek's January 2001 article in Microsoft Systems Journal “LIBCTINY, take II”.  It’s purpose is to reduce the sizes of executables and dynamic link libraries produced by Microsoft’s build chain.  All the files necessary to build and use TCLib, as well as the built binaries, are in the TCLib.rar attachment.

     That’s what it is.  But that doesn't explain why I created it or it's justification, or lack thereof.  I'd like to tackle that now.

     My first Windows coding was with Visual Basic 4 in the mid 90s.  Before that I had done QuickBasic and C in the DOS environment.  So I was a bit late in coming to Windows.  The install packages required by Visual Basic were large - in the several megabyte range, and I just assumed that's how it had to be in the more sophisticated Windows environment.  Until I discovered Charles Petzold and Win32 Api SDK coding, that is.  I was immediately attracted to this style of coding - perhaps because I'm inherently a 'low level' type coder, and I saw that binaries produced were once more back in the range of sizes I was familiar with in DOS.  By the late 90s when I acquired Visual C++ 6 from Visual Studio 98, the lower base line for Windows executables size wise was around 30k.  At about that time I discovered the PowerBASIC programming language and the PowerBASIC Community.  PowerBASIC was owned and coded by a rather well known compiler writer - Bob Zale, who had leased his code to Borland International in the 1980s, which corporation marketed it under the name Turbo-Basic.  After Borland's 'hey-day' when they could no longer compete with Microsoft in the compiler development space, Bob Zale bought back the rights to his code and started the PowerBASIC programming company and language.  Various folks in the PowerBASIC Community were using PowerBASIC to do Win32 Api SDK coding, and PowerBASIC translations of the code in Charles Petzold's Windows 95 book  were being done and made available.

     It was in this period in the late 90s when I was teaching myself Win32 SDK style coding using both C and PowerBASIC.  In my work life at that time I was using Microsoft's eMbedded Visual C++ 2.0 I believe it was for the coding of Windows CE apps for the handheld Windows CE based data collectors our folks used to collect field data.  At that time though - as was typical of those times, I was doing only C - not C++.  For the Windows Desktop applications I was working on though I used PowerBASIC - same coding style, as PowerBASIC's binaries were much smaller than Microsoft's.  I really liked Bob Zale's philosophy on coding - 'smaller, faster'.  He was from a time when sloppy coding didn't work.  Memory was insufficient for overhead, slop, or bloat in one's code.  To give an example of code sizes between PowerBASIC and C/C++, here is the most minimal Win32 SDK style program in C++ that can create a window using CreateWindow()...

Code: [Select]
// cl Form1.cpp /O1 /Os user32.lib
#include <windows.h>  // 88,576 Bytes

LRESULT CALLBACK fnWndProc(HWND hwnd, unsigned int msg, WPARAM wParam, LPARAM lParam)
{
 if(msg==WM_DESTROY)
 {
    PostQuitMessage(0);
    return 0;
 }

 return (DefWindowProc(hwnd, msg, wParam, lParam));
}
 
int WINAPI WinMain(HINSTANCE hInstance, HINSTANCE hPrevIns, LPSTR lpszArgument, int iShow)
{
 MSG messages;
 WNDCLASS wc;
 
 wc.lpszClassName = "Form1",   wc.lpfnWndProc   = fnWndProc;
 wc.hInstance     = hInstance, wc.style         = 0;
 wc.cbClsExtra    = 0,         wc.cbWndExtra    = 0;
 wc.hIcon         = NULL,      wc.hCursor       = NULL;
 wc.lpszMenuName  = NULL,      wc.hbrBackground = (HBRUSH)(COLOR_WINDOW);
 RegisterClass(&wc);
 CreateWindow("Form1","Form1",WS_OVERLAPPEDWINDOW|WS_VISIBLE,200,100,325,300,HWND_DESKTOP,0,hInstance,0);
 while(GetMessage(&messages,NULL,0,0))
 {
    TranslateMessage(&messages);
    DispatchMessage(&messages);
 }

 return messages.wParam;
}

....and here is the same in PowerBASIC Windows 10....

Code: [Select]
#Compile Exe  "Form1_PB.exe"            'Disk image: 6656 bytes   Memory image: 5312 bytes.
#Include "Windows.inc"

Function fnWndProc(ByVal hWnd As Long, ByVal wMsg As Long, ByVal wParam As Long, ByVal lParam As Long) As Long
  If wMsg = %WM_DESTROY Then
     Call PostQuitMessage(0)
     Function=0 : Exit Function
  End If

  fnWndProc=DefWindowProc(hWnd, wMsg, wParam, lParam)
End Function

Function WinMain(ByVal hInstance As Long, ByVal hPrevIns As Long, ByVal lpCmdLn As Asciiz Ptr, ByVal iShow As Long) As Long
  Local szClassName As Asciiz*16
  Local wc As WndClassEx
  Local Msg As tagMsg
  Local hWnd As Dword

  szClassName       = "Form1"
  wc.lpszClassName  = Varptr(szClassName)
  wc.lpfnWndProc    = CodePtr(fnWndProc)
  wc.cbSize         = SizeOf(wc)
  wc.hInstance      = hInstance
  wc.hbrBackground  = %COLOR_BTNSHADOW
  RegisterClassEx(wc)
  hWnd=CreateWindowEx(0,szClassName,szClassName,%WS_OVERLAPPEDWINDOW Or %WS_VISIBLE,200,100,325,300,%HWND_DESKTOP,0,hInstance,ByVal 0)
  While GetMessage(Msg,%NULL,0,0)
    TranslateMessage(Msg)
    DispatchMessage(Msg)
  Wend

  Function=msg.wParam
End Function
 

     So as can be seen in examining the above code it's very similiar.  Only thing is, the smallest I can easily make the C++ code in terms of the executable created is 88,576 bytes with Visual Studio's C/C++ compiler, whereas the same program coded in PowerBASIC 10 comes in 5,312 bytes.  The difference is even more stark than that, because with that 5,312 byte PowerBASIC executable there is full support for COM/OLE,  PowerBASIC dynamic strings from the OLE String Engine, and dynamic multi-dimensional arrays, and more .

     So for many years I coded in C or later C++ for my eMbedded coding because PowerBASIC didn't provide a compiler for that operating system, and I used PowerBASIC for my desktop work.  I might add at this point that the executables Microsoft's eMbedded C/C++ compilers emmitted were about the same size as PowerBASIC created ones - for the above program 4K or 5k or so.  Then disaster struck.

     Bob Zale passed away in 2011 at about the time he was just beginning to start work on an x64 PowerBASIC compiler.  It didn't happen and almost surely never will.  I needed a way to move into the x64 world of coding and PowerBASIC wasn't going to get me there.  So, for me, that meant Microsoft's bloatware producing C++ compiler from Visual Studio. 

     I had always fooled around with various compiler switches of both VC and mingw's g++ build systems to try to get smaller executables, but with varying degrees of success and failure.  My best luck had always been with the first versions of mingw, and with that I could get binaries down in the PowerBASIC range of sizes.  But then the 'powers that be' in the C++ world started morphing C++ into C# by wrapping more of the functionality of various operating systems into the C++ language, and mingw no longer did much better than VC in producing small executables.  For example, they started statically linking the pthreads library into the executables - bloating them by a factor of five.

     All of this made me very unhappy and I began in earnest to look for a way to minimize the sizes of my C++ executables.  That's when I discovered Matt Pietrek's Microsoft Systems Journal and MSDN articles about his Libctiny.lib and ways to minimize the sizes of executables.  All this work was from the late 90s to early 2000 time periods, and of course his sample code was for x86 binaries.  I tested it all, studied up on it all, and I found it truly worked.  Dennis Ritchie style Hello, World! console programs could be done in the 2k range using Matt's Libctiny.lib, and my C++ GUI template above could be done in 2,560 bytes, which was even half the size of a PowerBASIC executable!.  I had at long last found a possible solution to my problem, if only I could update Matt's code for UNICODE and x64.  To make a very long story short (it took me several years), I was finally able to accomplish that, and the result is my TCLib.   

     Interestingly, in all that work I found that whether I built as C or C++ had little difference, if any, on code size.  I recall when I was first testing Matt's code and had success in building a 2,560 byte Windows GUI program, I wondered if it could be done building in C++, and if my String Class would work with it.  It was around 2014 I believe when I first started working on all this, and by that time I was quite fluent with C++, and in fact had a pretty good String Class of my own which I used in lieu of the C++ Standard Library's String Class.  So believe me when I tell you I really held my breath when I made my first attempt to build even the most minimal String Class using Matt's LibCTiny.lib.  To my utter amazement and thrill it worked!  That's when I really knew I was onto something, and there was at least a chance of bringing this idea of mine to fruition.

     And that 'idea of mine' wasn't exactly what Matt Pietrek's idea of his Libctiny.lib was.  He only envisioned his Libctiny as being useful for a limited subset of his work, e.g., small utility type programs.  My idea for my TCLib.lib was for it to be the basis for an entire alternate development framework for C++ completely divorced from the C++ specific part of the C/C++ Standard Library.

     Having done C for a long time before I came to C++, and having done many basic family languages, I simply liked the way basic family languages did things like string handling, arrays, and file I/O.  I'd never choose to code an application in C if I had a basic family language alternative.  It's not that I didn't like C, it was more a matter of it simply taking longer to code any substantial application using that language.  In terms of the improvements to that situation that C++ was supposed to make, and could be found in the additions to the C++ Standard Library, I found they were, at least in my mind, awkward, and bloated code terrible. 

     So my solution to all of this was a somewhat eclectic approach where I'd use my TCLib to minimize the sizes of binaries produced by the Microsoft C++ compiler, and to substitute all my own library code for such things as a String Class,  dynamic multi-dimensional arrays, and whatever else I would need for the specific class of applications on which I worked.  For the type of work I do, which is forest biometrics, there were few in my profession - forestry, who used C++.  So it wasn't as if there was any body of commonly used C++ code out there that I would be denied from using by my abandonment of the C++ specific part of the Standard Library.  I had always coded my own algorithms anyway. 

     So that is where TCLib came from.  And I can't really justify it by saying that small code is better than bloatware.  With computer memory and storage where it is at at this point - I'd say nearly infinite, there is little merit to small code.  But coding for me has always been something of an intellectual art form.  For whatever reasons I simply am not satisfied with code that isn't tight.  If others want to take megabytes of code to accomplish some goal, and it doesn't bother them, that's fine with me.  But it's simply not the way that I do my art.

Patrice Terrier

  • Administrator
  • *****
  • Posts: 1980
    • zapsolution
Re: TCLib - Replacement For MS C Runtime To Reduce Program Size
« Reply #1 on: October 15, 2021, 11:07:41 am »
Fred

What to do to solve this error while trying to use NMAKE from a command prompt?

Microsoft (R) Program Maintenance Utility Version 14.29.30136.0
Copyright (C) Microsoft Corporation.  All rights reserved.

        LIB /NODEFAULTLIB /machine:x64 /OUT:TCLib.lib crt_con_a.obj crt_con_w.obj crt_win_a.obj crt_win_w.obj InitStdio.obj InitMath.obj crt_dll.obj newdel.obj alloc.obj alloc2.obj allocsup.obj  strlen.obj memcpy.obj  strcpy.obj strncpy.obj strcmp.obj _stricmp.obj _strnicmp.obj _strrev.obj strncmp.obj  _atoi64.obj atof.obj abs.obj memset.obj strchr.obj strrchr.obj strcat.obj memcmp.obj  atol.obj
Microsoft (R) Library Manager Version 14.29.30136.0
Copyright (C) Microsoft Corporation.  All rights reserved.


crt_con_a.obj : fatal error LNK1112: module machine type 'x86' conflicts with target machine type 'x64'
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\bin\HostX86\x86\LIB.EXE"' : return code '0x458'
Stop.
Patrice
(Always working with the latest Windows version available...)

Frederick Harris

  • Newbie
  • *
  • Posts: 47
Re: TCLib - Replacement For MS C Runtime To Reduce Program Size
« Reply #2 on: October 15, 2021, 03:50:59 pm »
Need to use command line console window configured for x64 builds.  There's what - 4 or 5 choices on the Start menu under Visual Studio for various command line windows.  I copy the shortcut to my desktop for "x64 Native Tools Command Prompt".  If you check out the properties for that shortcut it points to here...

%comspec% /k "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvars64.bat"

I think double checking on all this will solve that problem.  I've had that error a number of times.  Like I said, kind of, x86 obj files are going to be a bit different from x64 obj files, so the linker will balk on this, if it finds a mis-match.

Frederick Harris

  • Newbie
  • *
  • Posts: 47
Re: TCLib - Replacement For MS C Runtime To Reduce Program Size
« Reply #3 on: October 15, 2021, 04:53:54 pm »
Just reproduced that error using top choice from my Start Menu "Developer Command Prompt For Visual Studio 2019".  So you don't want to use that one!  I'm just assuming it's configured to target x86 tools for building x86 code.

crt_con_a.obj : fatal error LNK1112: module machine type 'x86' conflicts with target machine type 'x64'
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\bin\HostX86\x86\LIB.EXE"' : return code '0x458'
Stop.
« Last Edit: October 15, 2021, 04:57:21 pm by Frederick Harris »

Patrice Terrier

  • Administrator
  • *****
  • Posts: 1980
    • zapsolution
Re: TCLib - Replacement For MS C Runtime To Reduce Program Size
« Reply #4 on: October 15, 2021, 04:57:20 pm »
After i checked in

Test ->
      Processor Architecture for Any CPU project ->
          Auto
          x86
          x64
         


I had to delete all the existing .obj files, before i could create the  .lib file
Patrice
(Always working with the latest Windows version available...)

Frederick Harris

  • Newbie
  • *
  • Posts: 47
Re: TCLib - Replacement For MS C Runtime To Reduce Program Size
« Reply #5 on: October 15, 2021, 04:58:12 pm »
So you got it to work Patrice?

Patrice Terrier

  • Administrator
  • *****
  • Posts: 1980
    • zapsolution
Re: TCLib - Replacement For MS C Runtime To Reduce Program Size
« Reply #6 on: October 15, 2021, 05:10:07 pm »
Yes, but this is really not something easy to figure what to do, and i have to keep track of this thread for the next time ;)

I think that the best solution would be to create a DLL build in the environment to produce the LIB file.

By me, the size of the new TCLib.lib is 48102 bytes.
« Last Edit: October 15, 2021, 05:12:52 pm by Patrice Terrier »
Patrice
(Always working with the latest Windows version available...)

Frederick Harris

  • Newbie
  • *
  • Posts: 47
Re: TCLib - Replacement For MS C Runtime To Reduce Program Size
« Reply #7 on: October 15, 2021, 05:26:16 pm »
Quote
By me, the size of the new TCLib.lib is 48102 bytes.

I'm sure that's fine.  Just checked quick on my one laptop and I'm seeing 47,908.  I've noticed already it can vary some.  Don't know why.  I've also noticed that occasionally when I build a program with it there can be a 512 byte difference between one build and another, and that's the same exact code with no change.  Don't know why.

Patrice Terrier

  • Administrator
  • *****
  • Posts: 1980
    • zapsolution
Re: TCLib - Replacement For MS C Runtime To Reduce Program Size
« Reply #8 on: October 15, 2021, 06:22:18 pm »
Here is what to do to create a static LIB file from the environment.

Create a static library project
https://docs.microsoft.com/en-us/cpp/build/walkthrough-creating-and-using-a-static-library-cpp?view=msvc-160#CreateLibProject
Patrice
(Always working with the latest Windows version available...)

James Fuller

  • Newbie
  • *
  • Posts: 41
Re: TCLib - Replacement For MS C Runtime To Reduce Program Size
« Reply #9 on: October 15, 2021, 07:02:46 pm »
This is my structure for building TCLib. I need to roll my own to play nice with BCX.
You use a normal command prompt. I use File Explorer to navigate to my TCLibBuild Folder.
Then click the folder location name at the top so it highlighs, then type cmd.
It pulls up a normal command prompt in that folder for you.

I have a C:\TCLibBuild folder
In the TCLibBuild folder I have a TCLib folder along with build.bat and TCLib.mak files.
All my *.cpp and *.h source files are located in TCLib folder

In the TCLibBuild Folder I have a build.bat and TCLib.mak files.

This is my build.bat for using Visual Studio 2019  BuildTools or Visual Studio 2019 Community
Code: [Select]
@setlocal enableextensions enabledelayedexpansion
@ECHO OFF
@IF EXIST TCLib.LIB del TCLib.LIB
@copy TCLib\*.*
SET XTYPE= x64

IF EXIST "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Auxiliary\Build\vcvarsall.bat" (
  ECHO Using Visual Studio BuildTools 2019
  CALL "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Auxiliary\Build\vcvarsall.bat" %XTYPE%
  GOTO GotIt
)

IF EXIST "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat" (
  ECHO Using Visual Studio Cummunity 2019
  CALL "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat" %XTYPE%
  GOTO GotIt
)
GOTO issue
:GotIt
NMAKE /f TCLib.mak
del *.cpp
del *.obj
del *.h
GOTO done
:issue
ECHO could not find compiler path
:done
echo All Done

This is my make file. The file list is different than Fred's.

Code: [Select]
PROJ       = TCLib


OBJS = crt_con_a.obj \
crt_con_w.obj \
crt_win_a.obj \
crt_win_w.obj \
InitStdio.obj \
InitMath.obj\
InitStdLib.obj \
crt_dll.obj \
newdel.obj \
alloc.obj \
alloc2.obj \
allocsup.obj \
strlen.obj \
memcpy.obj \
strcpy.obj \
strncpy.obj \
strcmp.obj \
_stricmp.obj \
_strnicmp.obj \
_strrev.obj \
strncmp.obj \
_atoi64.obj \
atof.obj \
abs.obj \
memset.obj \
strchr.obj \
strcat.obj \
memcmp.obj \
tclver.obj \
version.obj \
atol.obj \
wmemcpy.obj \
stdinouterrW.obj \
stdinouterrA.obj \
ctypesW.obj \
ctypesA.obj \
memmove.obj
 


CC         = CL
CC_OPTIONS = /D "_CRT_SECURE_NO_WARNINGS" /O1 /Os /GS- /Gy /c /W3 /DWIN32_LEAN_AND_MEAN

$(PROJ).LIB: $(OBJS)
    LIB /NODEFAULTLIB /machine:x64 /OUT:$(PROJ).LIB $(OBJS)

.CPP.OBJ:
    $(CC) $(CC_OPTIONS) $<


James

Patrice Terrier

  • Administrator
  • *****
  • Posts: 1980
    • zapsolution
Re: TCLib - Replacement For MS C Runtime To Reduce Program Size
« Reply #10 on: October 15, 2021, 09:06:51 pm »
Thank you James, i shall try your solution if i couldn't find how to build a static LIB directly from the environment.
« Last Edit: October 15, 2021, 10:18:06 pm by Patrice Terrier »
Patrice
(Always working with the latest Windows version available...)