Today I started paying extra on my mortgage; about 180 bucks extra.

See, I haven't had any money goals recently (having met all the previous ones) and I only recently took up a new hobby which I'm saving up to buy something for. So I felt it was prudent to begin slowly picking away at the 176k; every little bit helps.

Concerning some work that I've been doing recently, I'll have available a comparison of different approaches I took to mass name resolution at work and the tools to back it up.

For a couple months now we've been doing mass DNS resolving of all our border netflow. My solution is able to resolve on the order of 500,000 addresses a minute. Note that this include duplicates. All told, it's about 24 gig of netflow a day.

We splunk all that, and have made special field extractions so that searching it is trivial. You may have noticed some of the charts that I posted a while ago which were relevant to these name resolutions.

Well, the plan is to start resolving more than just our border router. There are two problems though.

  1. We're not sure how much more the current solution can do before falling behind</li>
  2. We don't have enough cash to buy the necessary splunk license</li>
    #1 is solvable and is what I'm currently working on. I have no idea how we're going to solve #2 though. That's ultimately the problem with splunk. You can find so many uses for it that you run out of money before you run out of good ideas.

    That needs to be added to their product roadmap.

    The work that I've been doing for #1 involves 2 approaches.

    1. Distributing the work between nodes via TCP and HTTP</li>
    2. Distributing the work between a shared NFS volume</li>
      #1 is a PITA to make work correctly, however #2 isn't any walk in the park either. I think I have a solution for both though in the form of events in node. I've made a P.O.C. using idea #2 since it was easiest to wrap my head around at the time.

      I really like node. It's the "generalized event engine" that I was dieing to have. It's evented like bro, but it can be applied to so many more situations than bro can. On top of that, it compiles cleanly on linux and it doesn't have some special language that you need to write stuff in; it's all javascript. Thank you Ryan.