String Concatenation Speedup

đź’» Development
1

It’s probably a non-issue in practice, but since it’s really low-hanging fruit and at some extreme instances can give 10x (ten times) speed increase, You might want to use it. The speed-up is gained by reducing the amount of memory that is used for temporary strings. That in turn increases the probability that data stays at a CPU-chace and yes, You read it correctly: I use C++ style speed optimization technique to increase Ruby/JavaScript/PHP/C# code at least 2x (depending on data). It sounds crazy, but I just feel that it’s weird that at an era of the Internet, where a lot of data is in the form of text, XML/HTML/JSON/YAML/whatever-else-ML, auto-generated by software that generates HTML, exports XML/JSON, all those vast amounts of strings on the wolrd-wide-web are assembled sub-optimally.

suboptimal way:
    s_result="Hellow "+"World "+"and "+"Mars!"

leading to
    s_tmp_1="Hellow "+"World "  // length  7+6=13
    s_tmp_2=s_tmp_1+"and "      // length 13+4=17
    s_result=s_tmp_2+"Mars!"    // end result, so length does not matter
    // length of temporary strings: 13+17=30

optimal way:
    s_result=("Hellow "+"World ")+("and "+"Mars!")
    s_tmp_1="Hellow "+"World "  // length 7+6=13
    s_tmp_2="and "+"Mars!"      // length 4+5=9
    s_result=s_tmp_1+s_tmp_2
    // length of temporary strings: 13+9=22
    //                                   22 < 30

The differences at source code level between the suboptimal and optimal solution are:

suboptimal:
    s_result=""
    for(n times){
        s_result+=s_new_token
    }

optimal:
    ar=new Array();
    for(n times){
        ar.push(s_new_token)
    }
    s_result=s_concat_array_of_strings(ar);

The details and the implementations of the “s_concat_array_of_strings(…)” are at my GitHub repository. I do not remember, what license I used there, I usually use BSD licenses, but if You use MIT or “public domain”, then it’s OK. Just change the headers then.

For short mutable strings other solutions are faster, but just try out the test script and You’ll see. The same idea can also be used for other cases, where the operaton is similar to string concatenation. For example: 100 times 10 = 1000. Sample code for that can be downloaded from my comment at one Ruby related feature request. May be there’s some application of that idea, when You implement P2P code, where You need to concatenate pieces of downloaded files. As I demonstrated, the idea is useful even in scripting languages.

1 Like
2

@jan I’m not that familiar with the codebase. How much concat do we do? This may be neat.

4

Thank You for the compliment, but I guess that probably for You the neatest part of this whole story is that You really do not need an answer to Your question, because anyone can just download a single PHP/JavaScript/Ruby/C# file, rename the “s_concat_array_of_strings” analogue to “s_concat_array_of_strings_MyDogCatOrOtherPetName”, and use the function in his/her own module without ever telling others about it. Of course, placing the function to some project wide library folder for everyone to use would save the project from some clutter, but it is an option. :slight_smile: