Posted inUncategorized

CopperSpice Experiments – Pt. 16

seasoned_geek by seasoned_geek

Now we must create a minimal GUI “Hello World” to use testing out the theory. It will also be needed as a bug report test case if this turns out to be a bug rather than “should have had this switch on when building CopperSpice.”

In a terminal I did the following:

mkdir cs_gui_hello
mkdir cs_gui_hello/src
mkdir cs_gui_hello/resources
mkdir cs_gui_hello_debug
mkdir cs_gui_hello_build
cp diamond/resources/*.gif cs_gui_hello/resources

cd cs_gui_hello

roland@roland-amd-desktop:~/Projects/cs_gui_hello$ cp ../cs_hello/*.txt .
roland@roland-amd-desktop:~/Projects/cs_gui_hello$ cp ../cs_hello/*.md .
roland@roland-amd-desktop:~/Projects/cs_gui_hello$

In both CMakeLists.txt files I changed all cs_hello to cs_gui_hello and changed all Cs_Hello to Cs_GUI_Hello. I also updated the README.md. Then I renamed the build_info input file to: cs_hello_build_info.h.in and changed the important part of it to read:

#ifndef CS_GUI_HELLO_BUILD_INFO_H_IN
#define CS_GUI_HELLO_BUILD_INFO_H_IN

// Cs_GUI_Hello Version  "x.y.z"
constexpr const char *versionString = "@PACKAGE_VERSION@";

// Cs_GUI_Hello Build Date "08/17/2017"
constexpr const char *buildDate = "@BUILD_DATE@";

#endif

The copy of the *.gif earlier was just to get the animated gif files to test with. Since I really needed only one file I manually created cs_gui_hello.qrc in a text editor.

<RCC>
    <qresource prefix="/animations">
        <file>resources/spinning-red-diamond-4.gif</file>
    </qresource>
</RCC>

That means there was one other change in the CMakeLists.txt file in the src directory. The following lines had to be uncommented and a new source file has to be added.

Have to compile qrc to source and add it to source list

Now we open a terminal and test the assumption.

cd ~/Projects/cs_gui_hello_build

cmake -G "Ninja" -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=../cs_gui_hello_debug -DCMAKE_PREFIX_PATH=/usr/lib/cs_lib/lib/cmake/CopperSpice ../cs_gui_hello

ninja
ninja install

valgrind --tool=memcheck --trace-children=yes --leak-check=full --show-leak-kinds=all --log-file=/home/roland/hello.log ./../cs_gui_hello_debug/cs_gui_hello
What I see in CopperSpice

No, I didn’t run it in valgrind the first few times. I did towards the end because I wanted to point out a bad thing in the code. It is code like I see in many many places on the Internet. You are in a hurry and bang something out fast then create this kind of self replicating screw-up as many many people find and re-use your stuff because it “seems to work.”

The code

Yes, I deliberately pasted an image so you couldn’t just scrape the code and go! I need to paste a few more images so you have a frame of reference. I went to the bottom of the log file and searched backwards for “lost.”

memory leak one
memory leak two
memory leak 3
Repeating memory leak

Hopefully you can zoom in on the images to see the 10+K leak in the first image and the 700+ byte leak in the second followed by a pair of 500+ byte leaks. What is interesting about the last 512 byte leak image is that it repeats for every disk block read for the gif.

All of this happened because of line 19 (or so I thought). QMovie doesn’t provide a constructor with a non-optional parent pointer as the first parameter. If you want to create in one line much like I did at line 19 you have to provide an empty QByteArray then the parent.

Now we get to why I pointed out the qDebug() statement in the last post.

no supported formats

There are no supported formats. I am hoping there is just a switch needed for compilation of CopperSpice that was missing when I built. It would be odd for the documentation to be way ahead of the implementation.

I changed the code to see if parentage would fix the leak because you can’t really do much cleanup when the source ends with:

return a.exec();

I have had to do cleanup over the years. You end up coding something like:

int retVal = a.exec();
/*
 *  Do all of of your cleanup paying special attention
 * to the fact you no longer have an event loop so much
 * of the Qt cleanup won't work because a surprising amount
 * queues an event.
 */
return retVal;

I changed the code to read as follows:

#include <QApplication>
#include <QMainWindow>
#include <QLabel>
#include <QVBoxLayout>
#include <QMovie>
#include <QByteArray>


int main( int argc, char *argv[] )
{
    QApplication a( argc, argv );

    QMainWindow mw;

    QLabel *hello = new QLabel( "Hello World!" );

    qDebug() << "supported formats: " << QMovie::supportedFormats();

    QLabel *movieLabel = new QLabel();
    QByteArray emptyFormat;
    movieLabel->setMovie( new QMovie( "://animations/spinning-red-diamond-4.gif", empty_format, movieLabel ) );

    QVBoxLayout *layout = new QVBoxLayout();

    layout->addWidget( hello );
    layout->addWidget( movieLabel );

    QWidget *w = new QWidget();

    w->setLayout( layout );
    mw.setCentralWidget( w );

    mw.show();

    return a.exec();
}

Not only did I have all of the same leakage, there was new leakage for the QByteArray. This is the problem with these quick little test programs. Just a few lines of code gives you a whole lot of leakage.

Few developers take the time to actually run valgrind or any other memory checking tool. Using tools like this in an AGILE world is something that is “never in the sprint.”

To be fair, I went over to the other machine and ran valgrind on that little demo. There were lots of little leaks that seemed to all be in some Qt internal thing. Nothing I could really “fix.”

valgrind --tool=memcheck --trace-children=yes --leak-check=full --show-leak-kinds=all --log-file=/home/roland/hello.log ./gui-hello

The ideal situation is for your application, no matter how small, to never have the word “lost” on any of the blocks in your valgrind report. You won’t be able to get rid of “still reachable” without severely impacting performance. This generally happens because the compiler tool chain has its own memory management library helping new/malloc out. Instead of allocating RAM from the system it goes and gets a big chunk then doles it out per each request your application makes.

Sometimes there is even a switch you can add to the build command line to set the size. If you’ve done some stats or simply wrote a hokey little “free memory” slot hooked to a timer so you “know” about how much free memory their is during your test runs, you can set the initial allocation to be 

(TOTAL_MEMORY - Smallest_Free_Memory - Estimated_Binary_Load_Size)

This will gain you a bit of performance. On a desktop running on grid power with high quality RAM chips, you won’t gain a lot. On an embedded system using the lowest power RAM one could find to conserve battery life, you can breath new life into the application by finding and using such a setting. Dynamic allocation on battery powered embedded targets typically sucks. This is especially true when the system has no GPU for some reason.

mainwindow.h
mainwindow.cpp

So I created a MainWindow class and added it to my little gui hello test application. Yes, you see those deletes commented out. I ran the application with those deletes in place fully expecting a crash. A layout is supposed to become the parent of all non-parented things directly added to it. I should have died with a double delete error message but I didn’t.

I honestly thought deleting m_widget would delete m_layout. It should have in my mind. My first run at this deleted only m_widget and I still had the big leaks from the movie. Adding deletion of the layout got rid of the big leaks the movie was previously creating.

Leakage summary

Leaking 4K doesn’t sound bad until you realize the program only ran for a few seconds. If this was a medical device where you had 512MEG or less, I would be worried if the leakage increased over time. Something left on long enough leaking 3-4K every few seconds will rather quickly run out of RAM.

Most of this stuff seems to be things I can do very little about.

XCB leakage

I know the developers of CopperSpice are currently working on XCB things as they recently released some code for XCB. Probably haven’t gotten around to running valgrind on things.

QAccessible loss one
QAccessible loss two

Part of me distinctly remembers QAccessible leakage in Qt 4.x . The other part of me worries about the QString8 I see there because that is new with CopperSpice. The 697 indirect loss repeated quite a few times in the file. A large part of that may well be tied to this 504 byte indirect loss.

It is these sub-1K leaks that really bite you. People ignore them because they aren’t large. What people fail to realize is that they tend to happen often. I’m not picking on just these leaks. I’m talking about sub-1K leaks in every project with every tool set. It’s the end of the project and you have already run out of time so they just get ignored. Now you fail your ten day run-time test because your sub-1K leak happens often enough to consume all RAM.

fontconfig leak

The fontconfig leak most likely has to be kicked upstream to the libexpat people. XML libraries and tools are notorious for leakage. Honestly I’m surprised this one is so small.

Why am I harping on memory leaks? Because I’m currently working on an editor. I don’t know about you, but on Linux machines I’ve been known to leave an editor up for over a week. I mean I save everything before I go home, but many times I never shut down. I want the cursor to be right where I left it in every file when I come in the next day. Sometimes I get called away and the thing is up for weeks on its own. Even on my machines with 16Gig or 24Gig of RAM just how long do you think it can last if it has a leak, especially in a background thread?

<Previous-part Next-part>

Roland Hughes started his IT career in the early 1980s. He quickly became a consultant and president of Logikal Solutions, a software consulting firm specializing in OpenVMS application and C++/Qt touchscreen/embedded Linux development. Early in his career he became involved in what is now called cross platform development. Given the dearth of useful books on the subject he ventured into the world of professional author in 1995 writing the first of the "Zinc It!" book series for John Gordon Burke Publisher, Inc.

A decade later he released a massive (nearly 800 pages) tome "The Minimum You Need to Know to Be an OpenVMS Application Developer" which tried to encapsulate the essential skills gained over what was nearly a 20 year career at that point. From there "The Minimum You Need to Know" book series was born.

Three years later he wrote his first novel "Infinite Exposure" which got much notice from people involved in the banking and financial security worlds. Some of the attacks predicted in that book have since come to pass. While it was not originally intended to be a trilogy, it became the first book of "The Earth That Was" trilogy:
Infinite Exposure
Lesedi - The Greatest Lie Ever Told
John Smith - Last Known Survivor of the Microsoft Wars

When he is not consulting Roland Hughes posts about technology and sometimes politics on his blog. He also has regularly scheduled Sunday posts appearing on the Interesting Authors blog.