Name: Ilya Martynov
Member since: 2003-02-27 11:31:21
Last Login: 2007-10-31 13:09:36
Homepage: http://martynov.org/
Running Puppet on big scale
This is a rehash of my comment in slashdot discussion and my comment on Alexey Kovrygin's blog post.STOMP messaging for non-Java programmers on top of Apache ActiveMQ
Recently I was researching available options for messaging between Perl programs. In the past I had quite a lot of experience with Spread and I don't want to repeat. I hated Spread as it was buggy and unstable. So I looked into other alternatives: XMPP, STOMP and AMQP. AMQP has no Perl client so it was out. STOMP and XMPP are closely tied in my view but then STOMP looked simplier so I decided to go with STOMP. There is very good Perl client library for STOMP: Net::STOMP.
# send a message to the queue 'foo'
use Net::Stomp;
my $stomp = Net::Stomp->new( { hostname => 'localhost', port => '61613' } );
$stomp->connect( { login => 'hello', passcode => 'there' } );
$stomp->send(
{ destination => '/queue/foo', body => 'test message' } );
$stomp->disconnect;
# subscribe to messages from the queue 'foo'
use Net::Stomp;
my $stomp = Net::Stomp->new( { hostname => 'localhost', port => '61613' } );
$stomp->connect( { login => 'hello', passcode => 'there' } );
$stomp->subscribe(
{ destination => '/queue/foo',
'ack' => 'client',
'activemq.prefetchSize' => 1
}
);
while (1) {
my $frame = $stomp->receive_frame;
warn $frame->body; # do something here
$stomp->ack( { frame => $frame } );
}
$stomp->disconnect;
Erlang debugging tips
I've just started playing with Erlang so I have a lot to discover but so far I've found several things which help me to debug my programs:STL strings vs C strings for parsing
I'm working on a project where I need to build custom high performance HTTP server. One piece of this server is a parser for URLs in incoming requests. It is very simple and on the first glance it shouldn't be that slow compared with other parts of the server. Yet it was taking quite a lot of CPU according to the profiler. The parser is using STL and basically does several string::find() calls to find parts of URL. So I thought maybe string::find() is too slow and decided to benchmark it against strchr(). This is my benchmark code:
#include <string.h>
#include <string>
#include <time.h>
#include <iostream>
using std::string;
using std::cout;
int main() {
const char* str1 = " a ";
const string& str2 = str1;
const unsigned long iterations = 500000000l;
{
clock_t start = clock();
for (unsigned long i = 0; i < iterations; ++i) {
char* pos = strchr(str1, 'a');
}
clock_t end = clock();
double totalTime = ((double) (end - start)) / CLOCKS_PER_SEC;
double iterTime = totalTime / iterations;
double rate = 1 / iterTime;
cout << "Total time: " << totalTime << " sec\n";
cout << "Iterations: " << iterations << " it\n";
cout << "Time per iteration: " << iterTime * 1000 << " msec\n";
cout << "Rate: " << rate << " it/sec\n";
}
{
clock_t start = clock();
for (unsigned long i = 0; i < iterations; ++i) {
string::size_type pos = str2.find('a');
}
clock_t end = clock();
double totalTime = ((double) (end - start)) / CLOCKS_PER_SEC;
double iterTime = totalTime / iterations;
double rate = 1 / iterTime;
cout << "Total time: " << totalTime << " sec\n";
cout << "Iterations: " << iterations << " it\n";
cout << "Time per iteration: " << iterTime * 1000 << " msec\n";
cout << "Rate: " << rate << " it/sec\n";
}
}
ilya@denmark:~$ g++ -O3 test.cc && ./a.out
Total time: 0 sec
Iterations: 500000000 it
Time per iteration: 0 msec
Rate: inf it/sec
Total time: 15.5 sec
Iterations: 500000000 it
Time per iteration: 3.1e-05 msec
Rate: 3.22581e+07 it/sec
ilya@denmark:~$ g++ -O2 test.cc && ./a.out
Total time: 0 sec
Iterations: 500000000 it
Time per iteration: 0 msec
Rate: inf it/sec
Total time: 15.76 sec
Iterations: 500000000 it
Time per iteration: 3.152e-05 msec
Rate: 3.17259e+07 it/sec
ilya@denmark:~$ g++ -O1 test.cc && ./a.out
Total time: 0 sec
Iterations: 500000000 it
Time per iteration: 0 msec
Rate: inf it/sec
Total time: 19.23 sec
Iterations: 500000000 it
Time per iteration: 3.846e-05 msec
Rate: 2.6001e+07 it/sec
ilya@denmark:~$ g++ -O0 test.cc && ./a.out
Total time: 18.64 sec
Iterations: 500000000 it
Time per iteration: 3.728e-05 msec
Rate: 2.6824e+07 it/sec
Total time: 16.89 sec
Iterations: 500000000 it
Time per iteration: 3.378e-05 msec
Rate: 2.96033e+07 it/sec
Beyound XSS and SQL injections
What is common about HTML, XML and CSV files, SQL and LDAP queries, filenames and shell commands? All these things are based on text which is often generated by programs. And one commonly observed flaw in such programs is encoding rules are not being followed. These days many developers are aware about SQL injection and XSS problems as many books, online tutorials, blogs, coding standards, etc speak about them. Yet I'm not sure there is enough education so that developers use correct methods to protect their code from these problems. And besides this there is a lack of awareness that it is not just SQL and HTML. Definitely developers should think more broadly: if you generate programmatically any kind of text you must think about proper encoding of all data used in the generated text.The most common mistake committed by developers (and many security experts, I might add) is to treat XSS as an input validation problem. Therefore, I frequently come across situations where developers fix XSS problems by attempting to filter out meta-characters (<, >, /, “, ‘, etc). At times, if an exhaustive list of meta-characters is used, it does solve the problem, but it makes the application less friendly to the end user – a large set of characters are deemed forbidden. The correct approach to solving XSS problems is to ensure that every user supplied parameter is HTML Output EncodedA good example of wrong approach is PHP's invention called magic quotes. I have mixed feelings about this thing. On one hand it was probably a good thing because so many web based software is developed by dilettantes so overall we are living in a slightly better world as magic quotes do somewhat limit damage from bad code. On the other hand it teaches bad habits while not fixing all problems in bad code. Also it causes everybody else to suffer. Good news is that they are getting rid of this abomination in PHP6.
print join ",", @columnsWhat if one of columns contains say "," (comma)?
sprintf(cmd, "cp %s %s", orig_filename, dest_filename);Guess what happens if any of these filenames were not escaped for characters which are special for shell?
system(cmd);
IlyaM certified others as follows:
Others have certified IlyaM as follows:
[ Certification disabled because you're not logged in. ]
FOAF updates: Trust rankings are now exported, making the data available to other users and websites. An external FOAF URI has been added, allowing users to link to an additional FOAF file.
Keep up with the latest Advogato features by reading the Advogato status blog.
If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!