From network card to user-space in (almost) no time

Getting packets into userspace

Recently I was trying to get the vPath library to work in user-space.  What I wanted was to try it out before working on the production code, so I decided that I wanted it as a simple user-space program. The thing with exercising the library as it would be worked out in the real world code is that we need to get packets to it, make it do stuff and then return the packets back.

There are many ways to get packets into user-space, we have netmap, Intel DPDK, TUN/TAP interface, etc. (more can be found here). My first choice was netmap, the challenge with that is to get just a few packets into my program it sounded like lot of work. So I have put a pin in it to come back to it later.

Other way to do it would be to use tools like TCPreplay, wireplay, hping3, Scapy, Unix Raw Sockets, etc. in which we can craft packets like we want and send them to the target. But that was a no go too. Because even if I did have access to previous versions of TCP traffic, it’s nothing like the real world traffic. After a significant overhead the rewards are but a few in this case. Though these tools are like gold when it comes to recreating customer cases, for my current scenario, the overhead was too much for too little benefit.

To get a quick and dirty 5 min solution, I turned to TUN/TAP interfaces. These are neat little buggers, fitted the purpose nicely. You can read more about how to program them here.

As is the case in real world, I solved one problem just to get to the next one :-). Now the thing is even if we have TUN/TAP we need the real world traffic to come to it without any modifications. For the purposes of my exercise I needed the L2 header to be intact. That’s when after searching around for a solution, playing around with linux bridge and Openvswitch, I found out this neat little utility – netsniff-ng. And that was the answer to (almost) all of my problems.

Now the one feature of netsniff-ng I ended up using was its “—mmap”, and as per the authors it is “Useful to have raw access to network packet data in user space”.

The netsniff-ng website states – “netsniff-ng, a fast zero-copy analyzer, pcap capturing and replaying tool”, the only part I have used is the fast zero-copy analyzer.

This is the way I hooked up the tap0 to eth1 -

ifconfig tap0 up  ; /root/netsniff-ng/netsniff-ng/netsniff-ng –in eth1 –out tap0 –mmap –prio-high

Finally, the problem I did not solve -

I just wanted to exercise my state machine by feeding it traffic, so I was not really worried about sending the traffic back, though I did manage to send it back this way -

ifconfig tap0 up  ; ( /root/netsniff-ng/netsniff-ng/netsniff-ng –in eth1 –out tap0 –mmap –prio-high & /root/netsniff-ng/netsniff-ng/netsniff-ng –in tap0 –out eth1 –mmap –prio-high )

But I know it can be done in a better way. Thats slated for another day, another venture.


Image credit: Thomas HawkLicenseImage Link


Finding innovation

2 tweets got me thinking:


In today’s world when you can really build a prototype and get running all on your own, is it really hard to innovate ? Are really really good ideas very very hard to come by ?

I think not really, there is room for innovation, still and always will be. There will not be a time ever when a person can claim – “All the innovation is done”.

We will have innovations, where solutions from one dimension are brought into the other. Like the MakerBot industries got the 3D printing solutions from the enterprise and really pricy dimension to the dimension of generally available technology.

Or innovation like Navisens which will reinvent something like tracking a moving body by using all the normal sensors which are available in the common smartphone, location of device placement & some clever mathematics. They are moving solutions from the dimension of thought to the dimension of productization.

Or there will be innovation like that of Google glass, where the power of millions of minds will come to the aid of a single one.

Or something like Trailerpop, where somethings which were found in the dimension of competitive games and related rewards is being brought into the dimension of content discovery.

Innovation will always be around, it was before and it will be after. But what is lacking is the will to build, will to create and will to make ideas turn into products. It was this will which gave necessary power to the products to be drawn from one dimension to another.

And at the end – persevere and try try try :-) success will be yours !


Truth about programing – Many ways of doing things – each one of them is right !

Was reading the excellent book - Assembly Language Step-by-Step: Programming with Linux by Jeff Duntemann, in that he quotes from Rudyard Kipling’s poem “In the Neolithic Age“.

Seems really applicable to coding with any language:

But my Totem saw the shame; from his ridgepole-shrine he came,
And he told me in a vision of the night: —
“There are nine and sixty ways of constructing tribal lays,
“And every single one of them is right!”

Yes many ways to do things and in this case every single one of them is right !


Thinking like Superman

I was using a Superman episode on Netflix as my crunch-muse while coding one day. If I remember correctly was trying to get a driver to work or something, some low level crunch. But for a moment I was distracted by what was going on in the story.

Superman was up in the air above the city and fighting some bad guy and almost winning. He was about to deliver the final blow when the bad guy shoots a laser beam on the nearby dam and breaks it. Superman delivers his final blow and the bad guy goes off reeling into space. Now – the dam has broken and the water is about to flood the whole city. The water is rising in the nearby river flowing within the city and is about to rush in….. What does our man of steel do ? He carves out a new path for the excess water with the lass beam from his eyes and with his icy breath he builds a new temporary dam to stop further damage ! Superman saved the day yet again !

Now instead of enjoying this nice rescue and feeling good about it, it kind of plunged me into deep thought. I was thinking, if I was Superman and had all those powers, would I think of such radical solutions like building a temporary dam or creating a new Grand Canyon. No ! I would not think like Superman !

This is kind of what is bothering me at the moment. Will I think like Superman ?

Let me put it in different words, now I have a problem, say want to build a computational device which will be suitable for lets say playing games. It will be built from the ground up to play games and enjoy video content nothing else. Will I design something like the Xi3 Piston ? Which has radically changed the PC architecture ? And while doing that gone uber green ? Will I use something like FPGA instead of X86+code ? Will I use the normal IP stack or 6LoWPAN or Contiki or Zigbee ?

Can we change our thinking radically to not reinvent the wheel, to reinvent with the current tools and think that maybe the wheels should not be all round ? Yes for some cases they should be round, but can we use all our resources, think like Superman and radically change the world ? Can we ?

This is what keeps me awake at nights…… Do I think like Superman ?



The world is flat

No matter how you want to look at it memory is linear – atleast in this case. So whatever you declare, 2D 3D array – the memory is going to be linear. The way you reach the elements is your problem – u can either go one by one and till the end or use [] operator or use a formulae like -

*(aptrptr + (dimension_of_first_array * i + j)

In the case below – dimension_of_first_array = 4;

This is the code and the pics are DDD pictures of the memory.

int a[4][4] = {{10,20,30,40},{50,60,70,80},{90,100,110,120},{130,140,150,160}};

int *aptr[4] = {NULL};

int **aptrptr = a;

int **ptr = calloc (16, sizeof(int));

int arr[3][3][3]=
{11, 12, 13},
{14, 15, 16},
{17, 18, 19}
{21, 22, 23},
{24, 25, 26},
{27, 28, 29}
{31, 32, 33},
{34, 35, 36},
{37, 38, 39}


Mind Without Fear

Mind Without Fear

Where the mind is without fear and the head is held high;Where knowledge is free;

Where the world has not been broken up into fragments by narrow domestic walls;

Where words come out from the depth of truth;
Where tireless striving stretches its arms towards perfection;
Where the clear stream of reason has not lost its way into the dreary desert sand of dead habit;

Where the mind is led forward by thee into ever-widening thought and action— Into that heaven of freedom, my Father, let my country awake.

- a poem by Rabindranath Tagore