f13.net

f13.net General Forums => General Discussion => Topic started by: Morat20 on February 29, 2012, 06:55:20 PM



Title: C/C++ Question
Post by: Morat20 on February 29, 2012, 06:55:20 PM
I am stumped. My current job is taking a bunch of C/C++ code (specifically -- old code written by a guy who learned C from a C++ book, and then modified and extensively changed by a guy who knew C a lot better than C++ -- in short, a hodge podge that's half class-driven and half straight up old school C and all mess) build on Borland Builder and moving the project to compile on something newer (in this case, Visual Studio 2008). We're moving it because Borland Builder has some weird linker issue thing wherein code past a certain size triggers a massive linker error, so the poor coder working on it spends more time externing shit to get around it than he does adding/fixing it.

This has been...a challenge. Borland Builder is fucking weird in the shit it allows, and a lot of stuff that skated by with warnings is now "errors" (including stuff like multiple variable definitions and the like) and I'm having to do a shit ton of work. 95% of it relates to string handling and the fact that I'm also moving it to Unicode, but that's neither here nor there.

Getting to the problem -- I straightened out a giant heaping problem related to a set of cpp files that defined global variables, poiinters, and something else I can't recall -- moved them to headers, insured they were only included ONCE (as opposed to, you know, multiple times like they were. How the fuck that worked I cannot tell you).

In cleaning up the #include statements, I have finally gotten to something that looks like this

main.cpp
#include "global variables.h"
#include "useful definitions.cpp"
#include "example.cpp"


example.cpp
//Lots of Code
//if (this thing is true)
#include "common_code_block.cpp"   THIS IS THE BITCHING PROBLEM.
//else

common_code_block.cpp
variable_defined_globally = blah;  //IT CAN'T SEE THIS FUCKER, WHICH IS DEFINED IN GLOBAL_VARIABLES.H
blah + blah;
//more code.

(In short, common code block uses a global variable defined in a header file two files up)

See that #include there? It's a .cpp file that is JUST a chunk of code. Not a function, not anything. Just a goddamn chunk of code, sitting there -- ripped out from that if statement and thrown into a .cpp file because another module uses that same rather hefty chunk of logic. Not even functionalized.

That? That shit does not compile properly because of those global variables I mentioned upthread.

Except when "example.cpp" gets called, it can't see "global variables.h" (the code block that got yanked out uses some of the global variables) but CAN see the damn definitions. Or vice versa.  IT should be able to see it, but doesn't. (It certainly did under Borland Builder, but of course I had to do a massive reorg job because Borland Builder somehow multiply included global_variables.h and not only allowed it to compile, it somehow worked. They're not extern'd, either -- it should have made multiple instances of those damn things. And since it gave a warning, I guess it did. It just somehow worked.

Obviously I can't #include the global_variables.h file in the common_block.cpp file -- I get multiple definition errors for very good reason.

So my problem: I've never actually used a #include as a method of just, you know, exporting a giant chunk of logic. Frankly, it would have never occured to me to do something like that. I'd have made it a goddamn function and, you know, passed shit in by reference. But the previous guy (who did this like 10 years ago) just cut and pasted the damn thing from one file to another. (And this isn't the only place).

So why the heck can't common.cpp see the global variables? And is there an actual term for the process of yanking all that shit out turning 100 lines of raw, unfunctionalized code into a #include and just slapping it into the program like some giant-ass macro?

Maybe there's some awesome I'm not seeing here, because it LOOKS like the world's worst way to do this and is not biting me on the ass. I can, if I have to, cut and paste the fucker back. But it's called "common code block" for a reason, and my collegue (who maintains and expands this, where I am currently ass deep in porting it to work in a compiler that's, you know, made by a company that still exists and upgrades and supports it) doesn't want to have to modify it in a half dozen places each time he has to change something in that file (and it's sister files, also done in the same way).

I really don't have the time to properly functionalize it -- frankly, I don't understand the code well enough to do it with any speed and "getting it done" is the current order of the day, not "fixing bad decisions from the past". So unless there's some trick to this weird "treat an include statement like a giant macro" metho or some compiler flag, all I can think of to do is cut and paste it.

Which gets to the question at hand:

1) Does anyone know why that cpp file might not be able to see the global variables?
2) Was this weird ass #include thing a common trick or concept I somehow just never saw?
3) Is there a fucking term for it, so I can google the damn thing more effectively and answer this sort of shit in the future? Because I can promise you that googline "Problems with an include file we made by yanking a ton of raw code out and slapping it into a new file" has not seen much success.

I've spent WAY the heck too much time straightening out the rat's nest of the header files and compiler order of the basic common includes as it is. I'm about to beat my PC with a bat. I'm at my wit's end, and I hope someone can help. Or at least assure me that it's a stupid way to do it in the first place, so I don't feel bad about cutting and pasting the shit back and telling my colleague we can make proper functions later.


Title: Re: C/C++ Question
Post by: Samwise on February 29, 2012, 11:00:25 PM
1) Does anyone know why that cpp file might not be able to see the global variables?

If I had the code in front of me, I'd be checking the obvious stuff first -- can a statement right before or after the #include access the global variables, for example?  I'm wondering if maybe they aren't actually declared properly in the header and this #include thing is a red herring.  The #include is preprocessor and shouldn't affect scoping.

Quote
2) Was this weird ass #include thing a common trick or concept I somehow just never saw?

It's atypical, but AFAIK perfectly valid and should work the same everywhere.  #include really just means "effectively copy and paste this file's entire contents right here".  Usually you use it with header files to copy class definitions around but that's really just convention; you can do it with any arbitrary block of code anywhere you like.  I think.

That said, it's pretty weird and confusing.  The way I'd probably do it (besides the obvious of putting that code in a reusable function, if for no other reason so that it doesn't need to be redundantly compiled multiple times) would be to put that common code in a header file as a macro, e.g.:

#define COMMON_CODE_BLOCK  int foo = 1;\
int bar = 2;\
doStuff( foo, bar );\
etc...

and then #include that header file up top (where headers usually go) and stick the COMMON_CODE_BLOCK macro in my actual code.  Still kinda weird but less so than having an #include in the middle of a function.

Quote
3) Is there a fucking term for it, so I can google the damn thing more effectively and answer this sort of shit in the future? Because I can promise you that googline "Problems with an include file we made by yanking a ton of raw code out and slapping it into a new file" has not seen much success.

I'd google around the terms "transclusion" and "scope" and of course "C++".


Title: Re: C/C++ Question
Post by: Miguel on February 29, 2012, 11:08:59 PM
Yes it's a stupid (if not illegal) way to do this sort of thing.  #include just takes the text of the file and includes it verbatim into the referenced file.  You often see stuff like this from people with backgrounds in embedded programming, where your data and text regions hold only a few precious bytes and *every byte of code counts*.  Remember that each and every function call creates an activation record and a frame that gets pushed onto the stack:  if you only get 128 bytes of stack space, that's only 8 nesting levels before you are hosed (especially if you declare a lot of local variables, which also get pushed onto the stack).  Embedded C programmers have to learn all of these tricks when you're writing code destined for a $3 processor with severe limitations like this. But I digress....;)

You really need to see what the output of the preprocessor looks like.  For GCC it would be the -E flag.  Your compiler may be resolving the dependency chain in a recursive fashion which means the #if may include the code first, then resolve references locally before all of the #includes are done.  In any case, looking at the output of the preprocessor will tell you what your compiler/lexer see's after all of the preprocessor directives are expanded/processed.  It will probably become apparent that there is a scoping or order of evaluation problem that you can then fix.


Title: Re: C/C++ Question
Post by: Morat20 on March 01, 2012, 05:29:36 AM
Thanks. That'll help.

And yeah, the guy that did this originally was an embedded guy if I understand the code's history properly. One of the thing that annoys me about VS2008 is I'm not entirely sure what order it's compiling things in. That preproccessor flag should be a godsend.

And the obvious "check to see if the global variable can be seen prior to the #include block" did not, in fact, occur to me. I blame two days of frustrating detangling on that.

It's going on my list to "fix properly later" if there's ever room in the development schedule. If you're going to use global variables like that, at least create a class to hold them, make them static, and then you can just create an instance and use the values that way. As long as you remember to initialize it at the beginning of your code. :)

I appreciate the help. You've hopefully saved me several more hours of headaches.


Title: Re: C/C++ Question
Post by: Paroid on March 01, 2012, 08:11:48 AM
If I understand this code architecture correctly, main.cpp is using #include directives to include the code from the other .cpp files.  Are you getting errors when compiling main.cpp or example.cpp/common_code_block.cpp?  Any .cpp which is designed to be #included is probably not supposed to be compiled by itself, and would give errors.


Title: Re: C/C++ Question
Post by: ezrast on March 01, 2012, 09:55:09 AM
If I understand this code architecture correctly, main.cpp is using #include directives to include the code from the other .cpp files.  Are you getting errors when compiling main.cpp or example.cpp/common_code_block.cpp?  Any .cpp which is designed to be #included is probably not supposed to be compiled by itself, and would give errors.
I bet this guy's got it. What happens if you change the name of common_code_block.cpp to common_code_block.h?
(edit: or otherwise explicitly ensure common_code_block.cpp isn't being compiled as its own module - I think just changing the extension does that in MSVS but it's been a while)

Getting to the problem -- I straightened out a giant heaping problem related to a set of cpp files that defined global variables, poiinters, and something else I can't recall -- moved them to headers, insured they were only included ONCE (as opposed to, you know, multiple times like they were. How the fuck that worked I cannot tell you).
That can work just fine if your headers look like
some_header.h:
#ifndef SOME_HEADER_H
// all your include stuff...
#define SOME_HEADER_H
#endif

It should allow you to throw as many "#include some_header.h" directives into your code as you want without getting errors for defining things multiple times.


Title: Re: C/C++ Question
Post by: Amarr HM on March 01, 2012, 10:12:41 AM
Are you calling the global_variables with the scope resolution operator? You may have this but it wasn't in your pseudo code, should look like this global variables::variable_defined_globally. I'm assuming here they are static.


Title: Re: C/C++ Question
Post by: Samwise on March 01, 2012, 10:14:09 AM
That can work just fine if your headers look like
some_header.h:
#ifndef SOME_HEADER_H
// all your include stuff...
#define SOME_HEADER_H
#endif

It should allow you to throw as many "#include some_header.h" directives into your code as you want without getting errors for defining things multiple times.

Header guards are crutches for people who can't be bothered to think about their dependencies.   :grin:


Title: Re: C/C++ Question
Post by: proudft on March 01, 2012, 10:16:44 AM
Random shot in the dark - you might be having a name munging problem with C and C++ disagreeing about what to call the variables.   If that include file is processed in a C file before it hits the C++ you have, the compiler might decide to stick with C format for the rest of the compile and thus C++ can't see them.   Wrapping the definitions (well, really, the #include line in this case) in an extern "C" { } block within the C++ code will fix that.



Title: Re: C/C++ Question
Post by: ezrast on March 01, 2012, 10:27:27 AM
Header guards are crutches for people who can't be bothered to think about their dependencies.   :grin:
Ha. Knowing people think like this makes me happy I didn't decide to be a programmer. *shudder*


Title: Re: C/C++ Question
Post by: Samwise on March 01, 2012, 10:38:27 AM
Header guards are crutches for people who can't be bothered to think about their dependencies.   :grin:
Ha. Knowing people think like this makes me happy I didn't decide to be a programmer. *shudder*
Don't worry, people that think like me are in the minority.  

Most programmers just sort of roll their faces on the keyboard until it compiles.


Title: Re: C/C++ Question
Post by: Tarami on March 01, 2012, 11:52:01 AM
Most programmers think most programmers do that. :oh_i_see:


Title: Re: C/C++ Question
Post by: Morat20 on March 02, 2012, 05:58:20 AM
I spent yesterday playing "cut and paste" with the code and seperating out all the class definitions into proper .h and .cpp files. (Finding, in the process, an entire "common" file that consisted of four seperate classes, three of them as header declarations one as a full class declaration -- two of which used each other int he constructor. How the fuck Borland ever compiled this mess I don't know. I made seperate files for them, forward declared the classes, and moved on to finding all the goddamn method declarations and moving them into properly named .cpp files so I can see what the fuck is going on).

That top about the preproccessor block helped -- the problem, once I spent six hours cleaning up the code into something that could be traced by the human brain, boiled down to those damn massive global variable and pointer files. MSVC needed the global variable headers in those code blocks to handle that compilation set.

Including them, of course, throws a multiple definition and reinitialization error. Which in Borland is apparently only a warning. (And no, they're not guarded). So yeah, apparently they've been ignoring a zillion "X is already defined and declared, doing it again" warnings. How this even works is beyond me.

So I spent the last two hours of yesterday -- and will all of today -- taking the massive ass global variables (hundreds) and global pointers (also hundreds) and externing them in the .h file, initializing them in the .cpp file so the .h can be included as needed without allocating and defining the suckers.

I want to shoot the guy that made all this. In his defense, he was a Fortran guy whose only non-Fortran experience was embedded systems, and he wrote this ten years ago when learning C/C++ from a book. It's still a massive fuckup and I have to go tell my boss -- WHO IS THAT GUY -- that the problem is MSVC will not let you shoot yourself in the foot to the extent that Borland will.

It's not an easy task to explain that the conversion is taking forever because the code was written in what might, if you squint, been almost marginally acceptable 10 to 15 years ago, but that MSVC and I would expect g++ and practically anyone would scream bloody murder about.


Title: Re: C/C++ Question
Post by: Lantyssa on March 02, 2012, 07:03:18 AM
"Boss, it's spaghetti code."


Title: Re: C/C++ Question
Post by: Morat20 on March 02, 2012, 09:03:34 AM
HIS spaghettti code.


Title: Re: C/C++ Question
Post by: Lantyssa on March 02, 2012, 09:45:53 AM
Well, yeah.  But instead of tap-dancing around it, be honest.  You've got to spend a ton of time sorting out the code he wrote.  If he's unhappy with that, he should have written it better.


Title: Re: C/C++ Question
Post by: Samwise on March 02, 2012, 11:25:53 AM
If it's something he wrote while learning C++, he's probably perfectly aware that it's a fucking wreck.  God knows I feel ashamed when I look back at the stuff I wrote ten years ago.


Title: Re: C/C++ Question
Post by: Morat20 on March 02, 2012, 05:22:36 PM
This has been one of the most frustrating weeks of my career. I'm not sure I've accomplished ANYTHING other than confusing myself.

It's like every bad coding practice ever is coming home to roost -- Borland Builder must be a highly permissive compiler compared to MSVC, because this appears to be a structural problem.

I've managed to break out the classes. I've externed the global variables, pointers, prototypes, what-the-fuck ever into .h files and initialized them in the .cpp files. Except some of those global variables are arrays that later get called using sizeof() operaters, but the arrays are defined in the header as black sized.

There's ridiculous uses of anonymous enum types -- which I keep getting multiple definition issues with, so I've added code guards to everything. Which later led to a fun recurvise fuck-up in a class definition that had me banging my head against the wall.

Basically the problem is the design is for shit. Absolutely for shit. In trying to do all this "common" shared code, instead of making proper classes they went with a zillion global variables that are referenced everywhere, overwritten, defined again...apparently Borland doesn't even BOTHER checking a whole bunch of shit (or if it does, it came across as warnings that were ignored not errors that have to be fixed).

And I admit, I have NOT helped the situation by breaking everything apart and putting it back together in a sane way (you know, classes in their own files, etc). So I've been stepping on my own foot in the process.

Everytime I fix one goddamn thing I get ANOTHER set of "multiple definition" errors, or complaints that no constructor exists, or all sorts of weird-ass complaints that are of the fun variety of "This isn't the actual problem. This is a problem caused by a fuck-up upstream". And I get to try to figure out whether I caused this problem, or just uncovered it because the compile process moved forward.

I think -- and as usual, this occurs to me on Friday when driving home (you know, because I had to stop dicking around and think) that with the changes I've already made I *think* I have a possible solution. At least I can say that after this week, I'm very familiar with the way this piece of shit works.


Title: Re: C/C++ Question
Post by: Lantyssa on March 02, 2012, 07:33:23 PM
It's like every bad coding practice ever is coming home to roost -- Borland Builder must be a highly permissive compiler compared to MSVC, because this appears to be a structural problem.
I can't talk about newer versions, but Borland C++ 4.x was shit.  gcc using Cygwin would be a step up.  It's possible it was great for experienced hacks, but as self-taught with a background where I learned coding by experimenting, it was a terrible compiler choice.  Trying to convert code to compile elsewhere threw countless errors.  Usually it was easier to just rewrite it from scratch.

(Not that my early C code was all that well written either...)


Title: Re: C/C++ Question
Post by: Evil Elvis on March 02, 2012, 09:22:13 PM
Borland XE is shit too, although not nearly as bad as Borland 5.  Thankfully I only have to deal with it for a few small applications, but I loathe every minute of it.

There's a huge program at my work that displays so many warning messages - including those multiple definition warnings that the developers evidently think is just fine to ignore - that you can't compile all the projects in the group in one go; it hangs the IDE.

At least you're not so dependent on VCL that you're unable to switch to MSVC++.


Title: Re: C/C++ Question
Post by: Sheepherder on March 03, 2012, 01:50:49 PM
"Boss, it's spaghetti code."

"Like a yak stopped in the middle of the freeway to give birth."


Title: Re: C/C++ Question
Post by: Morat20 on March 03, 2012, 08:21:10 PM
"Boss, it's spaghetti code."

"Like a yak stopped in the middle of the freeway to give birth."
There comes a point, however, where you can't keep saying "Boss, this code is shit" without sounding like you're just making excuses. At least now I'm in the code the OTHER coder regularly uses who can attest that it is, in fact, kinda shitty.

Plus he can verify that they have indeed been ignoring eighty-three million "Multilple Definition/Reinitialization" warnings.


Title: Re: C/C++ Question
Post by: Soulflame on March 03, 2012, 11:18:19 PM
Can you reimplement it from the ground up?  It'd probably be easier than fixing something that old.


Title: Re: C/C++ Question
Post by: Morat20 on March 04, 2012, 07:34:22 AM
Can you reimplement it from the ground up?  It'd probably be easier than fixing something that old.
It's been discussed, but it doesn't fit into the development cycle.

Another module of the code, that I'm going to be responsible for, I am going to be slowly reimplementing -- thankfully it was done in an almost correct C++ fashion (I've already converted that module to MSVC) so I can fix it in place.

I think the module I'm currently working on might get slated for a heavy rework about 18 months from now. Maybe. Problem is, end users don't really see the effects of a major reimplemntation so it's hard to sell to them. I think the best we're getting is what I'm doing now -- tearing apart the files and restructuring them into proper C++ files and fixing the global variables/pointers/prototypes shit.

Plain fact of the matter is, what I'm doing is basically GUI work for some FORTRAN stuff. The fortran stuff -- seriously pricey, 4-figures per seat sort of data and analysis packages -- is the important stuff. The GUIs, the plotting stuff -- kinda just access tools. The fortran folks add new equations and models regularly, we try to keep up.


Title: Re: C/C++ Question
Post by: Jeff Kelly on March 05, 2012, 08:49:41 AM
Oh the joys of embedded coders. I can relate.

The problem is not really that programming an embedded system is somehow different, because it doesn't have to be. It's mostly that embedded development has for decades been the playing field of self-taught programmers/engineers and of a lot of people that came to programming not out of choice. They learned programming because what was once an entirely electrical, hydraulical or mechanical system designed by serious engineers has slowly turned into a small embedded computer box with lots of sensors and actuators attached to it. Yet, in most companies I know, the same electrical, mechanical and process engineers that once designed the purely analogue systems are still around and develop code for the embedded systems that replaced them, except they'd rather not do that.

A CTO for the global market leader in brake systems once said to me that "we design brake systems here, not computers" after I asked about the software development processes and expressed my opinion about how outdated they were. This from a guy that had to oversee the development of complex antilock braking control systems or electronic stability control systems with a serious amount of code attached to it and severe safety requirements and repercussions.

For most of my professional life I have been the sole computer scientist/computer engineer in the whole company, regardless of the company I worked for. The companies are run by engineers that never wanted to write code or develop software and just kind of slid into it. They treat it as an afterthought, for them the "real" system is the bits of electronic equipment and wires, they employ more people for hardware design and construction than they do for software and it shows. You always lose when you argue with them that you need a certain spec'd component because it would "make the software run better". The same mindset that did Nokia in in the end.

I've worked for companies that had "version/revisioning with CVS on the agenda for next year"  in 2007, who still had the majority of their code base in embedded assembly written for a compiler toolchain by a company that went out of business a decade ago, who "will get around to port it to C eventually".

Who never heard of static code checking, god forbid something as recent as unit testing.

That's why for example the automotive industry basically mandates an entire development process for every supplier that wants to do business with them.

There are only entirely reasonable (some might say "well, duh") things in there like

Gather requirements
Ensure traceability of requirements and code
do static code checking, oh and we've even made you a handy ruleset so you won't have to bother
Use a software development model, ANY ONE WILL DO, just use one, pretty please.
do testing
let the testing be done by different people than the development
document everything
use version control/revision control
use only certified tools

Granted they call it Automotive Spice because it sounds better, but still.

Yet I had to fight for every single one of those things in nearly every company I've worked for from small town business to global market leader. They usually only budge when you tell them that they kinda agreed to do those things when they entered the contract with the Car Maker and  if you read to them the kind of trouble they are in if they don't do it. (Like: you'll pay millions of Euros and likely lose your company kind of trouble)

That's why Software Powerhouses like Microsoft, Apple or Google run circles around entire industries if they decide to develop products for them.

If they ever decided to build In-Car-Control/In-Car-Entertainment systems then the majority of companies in that sector will go out of business in a heartbeat. It's like your side is still drunk on the invention of the rock attached to a stick while the other side's mechanized infantry rolls in to kick your ass. (Although if Civ is any indication I kinda want a phalanx in that case)


Title: Re: C/C++ Question
Post by: naum on March 05, 2012, 12:56:11 PM
I never did much C++ programming (other than building some "toy" programs and reading through introductory texts), but I did a great bit of C programming. Though most of my "C programming" involved puzzling out API docs and examining the esoteric "sample" programs that always seemed to feature some obscure function devoid of any pragmatic benefit. And most of my work was in data communication platforms.


Title: Re: C/C++ Question
Post by: Zetor on March 05, 2012, 09:48:41 PM
My company does hw/sw security evaluation, and there have been some truly :awesome_for_real: things I've personally encountered when going through (for example) mobile phone baseband software -- it's the 20th century all over again.

For example, in almost every project there was someone who thought using fixed-size buffers and copying raw unchecked user input into it is a splendid idea. The reactions from the developer when we sent the vulnerability list back to them was amusing too. "Oh, so you say that using a char[8] is bad and can lead to a buffer overflow? No problem, I'll increase it to a char[100] and we'll be safe for sure!" Never mind that user input copied into that buffer can be up to 65535 bytes long...


Title: Re: C/C++ Question
Post by: Morat20 on March 06, 2012, 05:27:12 AM
Finally sorted the fucker out. FINALLY.

I had to go ahead and shove the #include "blahdy-blah" stuff right back into the code. It simply wouldn't compile otherwise. Luckily, all that "common blocks" and "common shit" was like 95% actual class methods or went INTO class methods -- classes that were already common between modules!

So it's not actually adding any time to development, because there's still only the one copy.

I code-guarded the fuck out of everything, segregated everything into readable and sanely named and situated files, consolidated classes, and basically did the job as properly as I could without going much beyond "cut and paste". Then I changed entirely how the damn thing started up so it does so in a sane way, and was able to get everyone to see the globals and only initialize the suckers once.

Now all I have is the 8 million lines of string literals I have to surround with a Macro to make it unicode compliant. Yay, wxWidgets and your total failure to overload the = operator, because "wxString = "This is a wxString" is too goddamn easy when you can do wx String = wxT("This is a literal.") Admittedly, the latest and greatest version of wxWidgets I think does support that. But I'm working with very old versions.

Mind you, they're using three difference versions. My next job is going to be updating those damn libraries and fixing the code so at least everyone is using the same damn wxWidgets version.

Anyways, thanks for the help guys. Between you and frantic googling, I got this done in far less time than I'd feared.


Title: Re: C/C++ Question
Post by: Miguel on March 06, 2012, 07:56:53 AM
Oh the joys of embedded coders. I can relate.

The problem is not really that programming an embedded system is somehow different, because it doesn't have to be. It's mostly that embedded development has for decades been the playing field of self-taught programmers/engineers and of a lot of people that came to programming not out of choice.

That certainly part of it, but I think it goes deeper.

A lot of software engineers don't understand how what they write ends up as machine code.  They don't, for example, understand the ramifications of design and how it impacts the translation from code to the machine.

Take virtual methods for example.  It's a very basic concept, and any object oriented programming book/class covers the essentials of "virtualness" and other related subjects like covariance and contravarience.  However the decision to declare virtual methods has certain...side effects.  One such side effect is that you now have a greatly expanded virtual method table that gets glued on each and every instance so that the runtime type of the object can be unwound.

So you end up seeing stupid stuff like virtual methods sprinkled throughout the code almost as if done randomly, and pass by value being used in object methods, and you end up getting tons and tons of object copies happening.  Maybe fine on a regular PC with 8GB of RAM, but on an embedded platform with a small main memory and a tiny stack, certain death with almost impossible to debug side effects (ever seen a stack trace taken in gdb once the stack has overflowed, and gdb tried to symbolcate the resulting mess?).

The hard part is these things are hard to put in book format, because the resulting "rules" are highly platform and compiler specific.  But one really has to dig down at this level of detail when working on simple (yet powerful) embedded architectures.

Good thing is that I'm seeing more and more embedded programming courses being taught both at universities and online, and they are being taught by experienced computer engineers, who know to say "yes, this is what your CS book says, but here's *why* it says that".


Title: Re: C/C++ Question
Post by: Margalis on March 06, 2012, 08:08:22 PM
Everyone should be required to do some PS3 programming. (Or anything cell-based I suppose)


Title: Re: C/C++ Question
Post by: Murgos on March 07, 2012, 04:30:52 AM
Everyone should be required to do some PS3 programming. (Or anything cell-based I suppose)

Why not just say multi-core or multi-processor?  Why Cell in particular?  Is there something in that architecture you feel is going to be important for people to know for the future?


Title: Re: C/C++ Question
Post by: Margalis on March 09, 2012, 02:08:08 AM
Why not just say multi-core or multi-processor?  Why Cell in particular?  Is there something in that architecture you feel is going to be important for people to know for the future?

Cell is one shitty main CPU and a bunch of super fast specialized chips with no access to main memory and tiny dedicated memory you have to DMA to. It breaks vtables and really does not play well with object oriented programming.

The way the cell SPEs work is you feed them an input stream of data and they transform and output, similar to a GPU. (It's quite similar to general purpose GPU computing actually)

That forces you to think in some different and interesting ways. It means you need data structures that are packed carefully and laid out in contiguous memory, which has side effects like being incredibly cache friendly and super efficient in both speed and memory. Programming for cell is not like OOP C++ programming or really even C programming, at the fundamental level ideally you're thinking about how to package and transform data rather than about classes and methods. Basically the fundamental unit of C++ programming is a class and the fundamental unit of cell programming is a byte stream.

I don't know if that sort of stuff will be directly applicable going forward but it does expand the mind and things that work on cell will work efficiently on any architecture. For example on the 360 2 hardware threads share the same cache, meaning that if you can write an algorithm that works on packed contiguous data things will run a lot faster.

But because the 360 has one unified memory for the most part you just derp around and write loops that make a million virtual function calls.

It's just something everyone should experience, like functional programming.

Super nerd mode engaging: warning!

Say for example you were working on a system where each actor in your scene was going to sample some points in your scene each frame in order to retrieve harmonic spherical lighting coefficients that it can weight and lerp to end up with some environmental fake GI shit. To do that on cell you need access to those coefficients and a way to look them up to to look them up they are arranged in something like a quadtree / octree / sphere tree. So conceptually you want to take a bunch of positions and feed them into an SPE along with some sort of flattened oct tree or something that you can batch in from main memory in chunks. You also have to pay attention to stuff like byte boundaries.

So maybe you want to sort the actors in such a way that ones in similar parts of your tree end up together, then send an SPE those actors along with some flattened part of the tree that is relevant to them, then you send over the next positions and further tree parts, etc. And of course none of these things you are sending over are actual objects with handy methods like getChildAt() or getHSLCoefficients() but rather id numbers and implied structure (like a array-based heap) and such. And you have to think about how to batch your DMA calls and write algorithms where the SPE can do work while more memory is being DMA'd etc.

On the 360 you'd probably just write a for loop that was like:

for each actor
  tree->getCoefficients(actorPosition)

One thing it really forces you to think about it what sets of data you have in your application, rather than something like class hierarchy in which data is somewhat implicit.


Title: Re: C/C++ Question
Post by: Morat20 on March 12, 2012, 05:30:34 PM
So, I finally get to a working executable. This requires rebuilding the base libraries, oh, a zillion times because I forgot to set one compiler flag on one tiny module so of course it won't link properly.

The executable crashes, of course, instantly. I even know why.

Basically I've got this:

globals.h
extern string MyString[2];

And I use that everywhere.

I've got a class, call it:
MyApp.cpp
MyApp::OnInit()
"Do stuff"

My question is -- how do I initialize MyString = {String 1,String2}; I want it to be so I could just make a global.cpp file that had all the inits in it, and #include the sucker at the top of MyApp -- or hell, wherever.

Unfortunately, that's not working -- I get a lovely <bad ptr> when it goes to do the MyString={stuff, stuff} part. So, you know, it's got no actual space -- it's never initialized, it's just going to assign it.

I'm too tired to figure it out tonight, but if anyone sees the obvious I'm missing, let me know. I spent WAY too much time tracking down fucked-up linker errors. (Including one bug that is NINE YEARS OLD and only occurs with these libraries under MSVC -- I'm lucky as hell I stumbled across the bug report, since the libraries are -- you know -- nine years out of date and I'm not allowed to update them as part of this project).


Title: Re: C/C++ Question
Post by: Miguel on March 13, 2012, 03:41:13 PM
What type does "string" resolve to?  Is it char *? Or perhaps char[]?


Title: Re: C/C++ Question
Post by: Samwise on March 13, 2012, 05:04:55 PM
Somewhere outside of any function (compiled once) you have a:

string MyString[2];

that defines that array, right?  (As opposed to the "extern" statement declaring it that you include and therefore compile everywhere.)  I assume that if you didn't have that you'd get an error at link time.  That should be what allocates the space for the two "string" objects.  Could it be that it's the assignment operator in your string class that's barfing because it's trying to clean up a hunk of uninitialized memory (e.g. it's trying to delete() an uninitialized pointer)?

I'd be inclined to avoid the whole problem by making your two string dealy an object rather than an array.  Objects have constructors; arrays do not.   :awesome_for_real:  Then you can just be like:

StringPair MyStringPair;

and your StringPair constructor will get called at the right time and do the right thing.  I think.


Title: Re: C/C++ Question
Post by: Morat20 on March 13, 2012, 05:38:33 PM
Yeah, it's like this right --

globals.h
extern std::string myString[2];

globals.cpp
std::string myString[] = {"One", "Two"};
std::string test = myString[1];


MyApp.cpp:
#include globals.h
#include globals.cpp

MyApp::OnInit() {
//blah  blah
}

After dealing with some weird-ass MSVC stuff (I'd accidentally included MyString.cpp in the solution, which meant it was compiled into a globals.obj, which meant when it came time to #include it into MyApp it threw the usual "already defined" crap -- so I removed it from the solution, so now it's included in the preprocessed MyApp -- just like it should be.

Whole thing starts up, goes to MyString.cpp and then vomits all over EVERYTHING.

So I blew up the extern and just tried to declare and define a std::string array and assign it (IE: pretend I never externed it globals.h didn't exist).

Get this -- it supposely allocates MyString (I don't think it does -- I checked it at run-time in the debugger, and the allocation looks really wrong. I'm new to MSVC so maybe I just can't read it, but it honestly seems to be acting like a char * that's only getting the first string. That's the data I can see in the watch).

Then it goes to the test assignment -- get this, it assigns it correctly and crashes. Not "after" -- somewhere during the "std::stirng test = MyString[1]" assignment. A watch shows "test" is assigned the correct string, but I have a memory access violation -- one I can't trace other than looking at the assembly code.

I'm still thinking I'm doing something idiotic, because I spent most of my day in meetings and then about three hours dicking around with wxStrings (which are an object) only to find that it was having both an internal problem (wxString didn't like the non-unicode literal I was passing it) AND crashing at the end of the brackets on the assignment.

I've looked at the intermediate files -- it seems fine (although those are such a ratty mess in places). It doesn't seem to be a scope problem. It's like it's got pointers but no space.

It seems to be unrelated to the extern -- apparently I'm just doing something stupid and the array is never getting space allocated or never correctly aiming the pointer. I'm considering reducing it to an array of char *'s just to see if that fixes it.

I think my biggest problem is simply I don't code this way. I would never, in a million years, have written something this way so it's really likely I'm doing something boneheaded and not realizing it. I'd have just built a class called "MyGlobalCrap" and created a single global copy, and had everyone use that. (Well, that's a lie -- this program is practically begging for a top-down OOD -- I'd have done it that way in the first place, and it'd be nice and clean. It was just written by someone who only half-understood C and didn't understand C++ using some third-party libraries that were in C++ and so half his examples were object oriented and his books weren't, hence the resulting mess. Which then evolved over 10 years.)

About the only bright side of my day is I worked out how to eek out more time on Borland Builder (we're up against some dumbass size issue that means it simply won't compile and link correctly if your project is over a certain size) -- we're using some static libraries I can change to dynamic ones and free up the space that way if I can't get this converted in time for the delivery.


Title: Re: C/C++ Question
Post by: ezrast on March 13, 2012, 06:36:36 PM
I don't think you can initialize arrays of non-primitive types that way. It sounds like the program is constructing the string objects and then unceremoniously dumping C-style character arrays into the memory areas of each one. Try replacing the declaration with

std::string myString[2];
myString[0] = "One";
myString[1] = "Two";


Title: Re: C/C++ Question
Post by: Morat20 on March 13, 2012, 06:50:06 PM
Yeah, that tracks -- I think I tried that, and ran into basically a bunch of compiler bitching about how "MyString" was already definied -- it thought that:

std::string MyString[2];
MyString[0]="My String"

was actually creating another array (of size zero!) of Mystring.

I may just scream "fuck it all" and create an function that initializes every goddamn variable in the entire damn thing and sneak it into the constructor for the base class. I'd prefer to do it in the cpp, but beggars, choosers, etc.

This is what I get for not coding in C for a decade. I keep thinking in C# and Java terms.


Title: Re: C/C++ Question
Post by: Samwise on March 13, 2012, 07:10:23 PM
Yeah, you aren't allowed to do assignments outside of a function like that.  You declare your globals outside of a function but you can't put arbitrary code out there.  That would be madness.

I believe that if your global is an object, the object's constructor will be called (before main() I guess?), which is why I suggested using an object rather than an array.  But initializing stuff in a function that gets called before anything else is good too.


Title: Re: C/C++ Question
Post by: Morat20 on March 13, 2012, 07:43:44 PM
Eh, I'll just sneak it into the common function .cpp, make sure it gets called first. Half the globals are pointers which don't need initialization other than being set to "null" (they're actually initalized to real values in the code, or else the entire mess would have exploded years ago) and what's not pointers are either primatives (a small handful, mostly stuff that should be consts but aren't) and strings used to populate various GUI menus and whatnot.

The whole thing's held together with spit and bailing wire, although at least now it's somewhat sanely organized.

The guy who has had the unenviable job of maintaining and adding to this rat's nest (who, in his defense, didn't design this mess and I've found his code segments to be generally very good, if obviously unfamiliar with OO principlies) would flip right the fuck out if he knew how much easier his job would be if he was allowed to shitcan the mess and rewrite it.

Especially if I could give him a quickie class in OO design and made him use C#. None of the GUI elements need anything exotic -- hell, even the plotting is done by gnu-plot and all the math is done by fortran. It only needs two coders (instead of just him) because the output from the engineers and demand for new features is just two much for one guy.

It physically hurts to look at some of the code and see how much easier, smaller, and more readable it'd be with a good design.

It's the part of the job I really, really hate -- seeing an old piece of code that's evolved, and really needs to be stripped and rebuilt to fit it's current needs and usage, which are considerably different from how it was originally designed. Except it's really hard to get management to sign off on "We'd like to, you know, have all your engineers stop making new shit we have to add and have the customers just take the next cycle off from expecting new features or better models. We'd like to rewrite it. So version 8.0 will look exactly like 7.0, but we'll be a lot happier with it from a working perspective".


Title: Re: C/C++ Question
Post by: Margalis on March 14, 2012, 04:22:04 AM
Allocating stuff in static code is something I always avoid because C++ static operation order is indeterminate and if you have your own memory manager or something (which you ALWAYS will on a console) you can allocate stuff before those are set up. Doing stuff in some actual method is almost always a better idea than trying to do it "before" the program starts.


Title: Re: C/C++ Question
Post by: Quinton on March 14, 2012, 05:02:34 AM
A few random things to try.  I'm far more used to gcc/g++ on the commandline than dealing with VisualStudio, so these may not translate 100%, but here they are anyway;

- #including .cpp files is generally a bad habit -- your makefile or IDE project should list all the .cpp files
- turn on ALL the warnings (-Wall for commandline compilers) - C and C++ often assume you know what you're doing and will consider something that other languages would rate a fatal error to be only warning-worthy... and by default not all of these warnings are enabled.
- when you think your project/makefile may be causing headaches, try making clean (blowing away all intermediate and final output)
- be sure to include the correct include files for the classes you're using -- #include <string> before using std::string, etc -- some compilers will let you get away with not including the necessary headers but end up generating incorrect code as a result
- Margalis is right -- static initializers in C++ are dangerous -- by spec they happen in an indeterminate order -- if you have global state, have an init() or some init_foo()/init_bar()/etc functions you call before doing anything else that sets it up sanely.

I tried reproducing your issue with a little test program, but couldn't (but again, on Linux-x86-64 w/ g++...).  It may be possible that if your header refers to std::string foo[2]; but it's actually defined as std::string[] = { ... }; that you're running into a situation where some files think they're dealing pointers of different types.  I didn't think modern C++ had this issue, but it might not hurt to ensure that apart from the extern, the way you declare the globals in your header matches the way you declare them in globals.cpp exactly.


Title: Re: C/C++ Question
Post by: Lantyssa on March 14, 2012, 07:12:47 AM
Then it goes to the test assignment -- get this, it assigns it correctly and crashes. Not "after" -- somewhere during the "std::stirng test = MyString[1]" assignment. A watch shows "test" is assigned the correct string, but I have a memory access violation -- one I can't trace other than looking at the assembly code.
Sounds like a typical memory allocation error.

I'm not familiar enough with C++ to know if that line should work as-is or not, but you'd see that if your string wasn't null-terminated or if you had a pointer you were using to write somewhere you shouldn't.


Title: Re: C/C++ Question
Post by: Morat20 on March 14, 2012, 11:01:40 AM
Yeah, I'm going with allocating/initializing this stuff in a method or function. It'll just make my life easier. In deference to my colleague having to relearn where I put everything, I'll follow the same naming convention as the files...

Ugh. That's what I get for trying to minimize change as much as possible, rather than doing it the way that's right.

And yeah, you're right about C/C++ letting you shoot yourself in the ass. The only reason this code was such a mess was that Borland C++ considers "redefinition" and "reinitialization" of variables as a warning, not an error.


Title: Re: C/C++ Question
Post by: Miguel on March 14, 2012, 04:42:42 PM
Some compilers have very odd scoping rules, especially if you mix in "extern "C" blocks.

Probably what is happening is of the following lines:

std::string MyString[2];
MyString[0]="My String"

The second line could be allocating an object MyString of a different type, assigning it to a name-mangled variable MyString, then discarding the object immediately (especially if the compiler cannot locate a reference within the current scope to a matching object).  The compiler may even be clever enough to completely remove the code as it has no WAR or WAW side effects.

You'd have to look at the assembly source in order to be sure.


Title: Re: C/C++ Question
Post by: Morat20 on March 14, 2012, 05:17:23 PM
Some compilers have very odd scoping rules, especially if you mix in "extern "C" blocks.

Probably what is happening is of the following lines:

std::string MyString[2];
MyString[0]="My String"

The second line could be allocating an object MyString of a different type, assigning it to a name-mangled variable MyString, then discarding the object immediately (especially if the compiler cannot locate a reference within the current scope to a matching object).  The compiler may even be clever enough to completely remove the code as it has no WAR or WAW side effects.

You'd have to look at the assembly source in order to be sure.
That fits with what I'm seeing. Initializing it in a function should fix that, since the compiler will throw a hissy if it can't find the matching global variable.

extern std::string MyString[2];

#include "globals.h"
void init() {
MyString[0] = "String 1"
MyString[1] = "String 2"
}

It'll either resolve that to the global or scream at the compiler stage, I won't have some weird run-time allocation nonsense.


Title: Re: C/C++ Question
Post by: Miguel on March 15, 2012, 10:47:39 AM
Just to make you feel better (or worse, in this case), the following code compiles and runs fine on g++ version 4.4.5:

(Output of preprocessor with -E switch):

Code:

# 2 "main.cpp" 2


# 1 "globals.h" 1
extern std::string myString[2];
# 5 "main.cpp" 2
# 1 "globals.cpp" 1
std::string myString[] = {"One", "Two"};
std::string test = myString[1];
# 6 "main.cpp" 2

int main( void )
{
    std::cout << myString[0] << std::endl;
    myString[1] = "Reassign";
    std::cout << myString[1] << std::endl;
}
backend-new test # g++ -Wall -pedantic main.cpp -o main
backend-new test # ./main
One
Reassign
backend-new test #

My gut tells me you are getting into some weird late-binding problem specific to MSVC.


Title: Re: C/C++ Question
Post by: Morat20 on March 15, 2012, 08:54:05 PM
Yeah. It appears to work fine for primatives (bools and ints, mostly) but not std::strings and wxStrings -- and the latter are objects.

Not arrays, not simple assignments. However, functionalizing it:

globals.cpp
wxString MyString[2];

void init() {
MyString[0] = "This";
MyString[1] = "That";
}

Appears to work fine -- at least it's compiling, linking, and moving further in the execute process. Sadly I had SUCH a mass of strings, arrays of strings, and the same of wxStrings that I spent most of my day today moving them to the relevant init() functions and still aren't done.

However, the fact that it resolves MyString and doesn't throw a link error or a compile error is heartening. It's scoping correctly -- it'd scream a hissy if it wasn't assigning to the globals (there are no other variables with that name in scope, and it's not creating multiple definitions either -- it screams about that too).

I'll be satisifed when it actually executes far enough to RUN the damn init() functions and assign them, but I think it'll work fine.

I spent part of today trying to work out how tough it would be to move the Borland Builder wxWidgets stuff from the static libraries to a dll, only to get frustrated at the bullshit that is Borland Builder. Making the dll was easy enough, getting the motherfucker to link to it has proven to be challenge.

Goddamn assert failure. We're moving to the new compiler because of this, and I was hoping by switching to dynamic linking rather than static linking I'd basically free up enough space for the compiler to hack this latest build, giving me another few months to finish this conversion -- rather then have everyone breathing down my neck about why it isn't finished.

It's not finished because (1) it's a giant motherfucking program, (2) It's not written Unicode-compliant, (3) It's not written to a sane ANSI standard and (4) People ignored 83 million warning messages that should have sent up screaming red flags that MSVC won't ignore.

They're lucky I got an executable this fast, even if it crashes. I hadn't even glanced at the code before this got dumped in my lap, and until three weeks ago NO ONE was saying "Oh, and Morat, we can't do the delivery until you finish the conversion then port in all the new code your counterpart is writing as you do it".

OTOH, my boss has stopped asking for regular status meetings and seems to now be reasonably assured I am competent, productive, and not wasting time. I've been with the group maybe 6 months, and I've been doing this sort of impossible to benchmark/estimate completion time shit since I got here. Since I've finally moved to code the OTHER GUI guy knows intimately, and can discuss with him at length what I'm having to do and why, I'm guessing he can finally give my boss some actual feedback.

It's not the bad "I've stopped asking because I'm replacing him" type of stopped asking. It's the "I can actually tell he's doing stuff and I'm going to leave him alone to do it" sort.

I can't blame the guy -- it's hard to assess whether I'm doing the job well, or in a timely manner, or anything like that when no one can give you firm estimates of anything. Hell, I'm the only one there that really understands what "extern" means, or how a linker works. It's not like they're all old-school "I work from makefiles" types.

Besides, either I produce or don't. *shrug*. The alpha build is going to be buggy as all fuck though. I'll get them something executable in time, but it'll be until beta or later before I've ironed out 95% of the conversion bugs. Too many code paths, and too many changes to do otherwise.