Apr 142012
 

Permalink - writemycode.net

In the spirit of a very similar blog post, I decided to expand upon a specific area of that article…

Consider a question like this:

I want to do this. Create 10000 files, (filename can be just combination of time and random number). File size should be 4k. And I want to time this. say how many seconds it will take.

How can I do this on bash?

Thank you.

Obviously this person needs some assistance, and the question is very short and easy to understand. But the problem with this type of question is that there are really only two ways someone can approach the answer. Your options for an answer are either to just give the person the complete code snippet as they have requested, or fall back on a lengthy personalized tutorial.

Short questions that primarily request a complete code example as an answer are counter-productive to code communities.

Answer Option 1

As the topic of this article suggests, option 1 is “Write my code for me”. It might seem easy as a one-off situation to simply donate a working code snippet and get the person asking this question moving on their merry way, but really this is not helping them in the long run. They haven’t learned anything beyond the process of hitting a roadblock, then immediately going online to ask for a solution. Had this person included some references to what they have researched, and most importantly a code snippet representing what they have attempted, viewers of this question would have a basis for comment and potentially an answer pointing out where the person has gone wrong.

Answer Option 2

And in the other direction: providing a lengthy tutorial. We all want to help and teach, but this is just one of numerous questions floating out in the ether that requires a response. Can we really spare that much time for every single question like this to re-teach material that is most likely already documented in generalized contexts all across the internet? That would create quite a lot of redundant information simply because each person combines new components into a new question needing a new lesson. Really, every part of this question can be Googled quite easily. Why ask a community to do a new custom writeup for you?

Now, assuming we actually wanted to write a tutorial for this person asking the question. The problem at this point is where do we begin? The information provided doesn’t suggest that this person has a grasp on any part of the problem. So the tutorial answer might need to include:

  1. Bash and for-loops
  2. How to get the current time
  3. How to generate random numbers
  4. Creating files
  5. Populating new files to a specific size
  6. How to write a complete bash script, and time its execution.

Had they told us what they know how to do so far, and what aspect has them stuck, we could simply focus on one area and provide a good bit of knowledge to get them moving again. But right now this is just too much work to net a situation where they will learn something.

For those that are immediately inclined to provide the complete code snippet to solve the problem, where do we draw the line? What if the question being asked would need 10 lines of code? 20? 100? And if you are also interested in frequently helping people, would you be willing to provide 5 lines of complete code to 10 people a day, knowing that each person probably didn’t learn much? Furthermore, after having given this individual a quick answer, you have now rewarded their lazy behavior, and more than likely just encouraged them to repeat the bad habit again.

Through a conversation between my coworker and I, some interesting metaphors were raised that I simply can’t resist from sharing…

Vending-machine communities

Put a question in the slot and pop out a solution.  
Be it a traditional forum, an online discussion group, mailing list, or a trust and reputation based technical site like stackoverflow.com, these communities are driven by people, not machines. People have to take time to review content, and contribute their knowledge. We all work hard to acquire that knowledge, so lets all try and put some value on it in the form of the quality of our questions. A .25 cent answer is insulting. Treat your questions like they are costing you actual money that properly reflects the value of people’s knowledge. Code communities don’t work for you, and you don’t work for them. We are all here to help because we love it. Please don’t make us hate it, or feel like we are all just part of a big vending-machine.

Toilet paper answers

Answers that can be used only one particular time for one particular situation.
There is an insane amount of content on the internet. It’s hard enough sometimes to sift through the results of a vague Google search, let alone the content on our individual communities. When you ask a question that provides zero context, or proof of the extent of your current effort, then both the question and the answer are for the most part “throw aways”. If its going to be a persistent part of a community space, it should aim to benefit future support-seekers with similar situations. Referring to the example question above, someone going with Option 1 (“Write my code”) will end up providing an answer that will likely not help many people beyond this situation. Unless they too are looking for a way to do a for loop and create 10,000 4k dummy files and measure the execution time. The only way it would stand to benefit future visitors is if the answer did Option 2 and wrote a fully self-describing tutorial.

Conclusion

I can only speak for myself about what I might do. I consider myself to be the type that would go as far as to look at API docs for someone, and work out some pretty extensive examples on my local machine. I might even install libs that I don’t have or have never used before in an effort to provide assistance. But I need to be motivated to do so. It’s exciting for me to write out a page-length of information if I know it will help this person. But in a case like the above there is no show of effort and no context provided — just a person asking to have code written for them.

As I suggested already, we all want to help. Thats why we frequent these forums and sites. But we help these types of answer-seekers even more by withholding instant gratification.

And now… I direct you back to whathaveyoutried.com

 

Jul 282011
 

Preface

I’ve been programming in python for over 5 years now, and I love the language a lot. I would look for any opportunity to accomplish my coding tasks using python, as opposed to learning new languages. PyQt4 for user interfaces, Django for web design, etc… And when python just isn’t an option, I have done my share of php, javascript, objective-c and so on.
Recently I started thinking that I should probably expand my programming knowledge a bit to make myself more marketable. While python is pretty key in the world of Visual Effects pipelines… so is C++. Having previously focused on being a Compositor and not a programmer, I had always told myself that if I needed to learn C++, then I was going too far in that direction. But now that I have been working in pipeline development for over 2.5 years, it has become clear that I really should explore the world of compiled languages.

Grabbing some books on C+, I dove in. Yuk. Why is it so freaking boring? It really does suck trying to learn C++ after having been so spoiled with python for so long. Its like taking a Aborigine and trying to turn him into a snooty English gentleman. Hmm…Is that right? Well whatever. Its completely disorienting. So many things that I never had to think about, like type declarations, memory management, pointers… But I kept on reading and learning.

Then I came across a post on Google+, by someone that actually works at Google, mentioning a language being developed in-house. So I started reading about Go @ http://golang.org/

The Go Language

From the standpoint of a python programmer, Go feels like it sits right between C/C++ and python. You get the simpler syntax, but with the speed of a compiled language. Because Go has garbage collection, memory management isn’t a concern. I was never used to the code/compile/test/repeat pattern before, but it compiles so fast and is so easy to set up a project that it feels pretty natural. Go doesn’t require you to have header files and declare everything in advance, so banging out a simple program is quite fast and only slightly more overhead than writing a python script. Just have to add the step of compiling it. As far as the library, so far I have found everything pretty useful. And it seems that every time I search for a Go binding for something, I find one. I will go into more detail on that in a bit.

One thing that will make python programmers feel more at home is the type inference while creating a variable. While you can do something like this in Go:  

var myString string = “Foo”

you can also type the same thing like this:  

myString := “Foo”

:= operator lets you initialize a new variable and makes the compiler figure out what type it should be, based on the return type of the right hand side.

Pointers are still kind of strange for python programmers, but Go is a lot more flexible about using them. You don’t always have to explicitly dereference them like   (*myPointer).myFunction(). It just does it for you when you access member functions and attributes:  myPointer.myFunction().

Having no type inheritance is also a bit different as well. Instead of creating base abstract classes, and subclassing them, you only have structs… no classes. But you share functionality by using interfaces. An interface is just a definition of methods. If any object implements those methods, its consider that type of interface. This is something I have yet to really get into, since my current first project is more of a cmd program, rather than a pkg library. I’m sure I could be using interfaces already, but it hasn’t quite felt natural enough to incorporate as of yet.

A pretty crazy aspect of Go is its native support for concurrency using what they call “Goroutines”. The closest way I have been able to compare it to my experience in python is while using ZeroMQ for messaging. ZeroMQ promotes not only using its library for messaging, but also to replace issues with threads and locks. It has similar concepts in promoting concurrency. You divide your program up into its components and instead of sharing data structures between them, you communicate over channels (sockets in ZeroMQ). When you fire off a Goroutine, you aren’t waiting for it anymore. It can run and do its thing and you keep on going. You can then send data back and forth with channels, and even use them just for signaling, like saying “ok now exit”.

Actual Usage

I’ve been writing a message server so far in python, using the Tornado web server, along with some socket.io bindings called TornadIO, and also ZeroMQ for internal communication. So far its been working pretty well, but there are a lot of complex layers, with ZeroMQ sort of riding on top of Tornados ioloop. I decided to try and rewrite this server in Go. Turns out Go has a lot of built-in support for doing exactly what this python server was doing. The whole web-socket server functionality is part of the standard http library module. I quickly found Go bindings for socket.io and I was on my way.

It was quite fast to get the server to the point of doing global messages, but now I had to think about implementing the support for channel subscriptions. My first instinct was to go grab ZeroMQ and its bindings again, or to use Redis for the messaging, but then I was thinking “Shouldn’t I be able to do all this with Go’s concurrency?”. One thing that really helped me out was how fast everyone responds on the golang-nuts discussion group. It was quickly pointed out by more than one person that I should definitely be able to accomplish the internal message routing purely in Go. And they were right. I just set up a “dispatcher” function and run it in a loop as a goroutine, and then pass messages in and out of it. The dispatcher manages its data structure, and no other part of the code accesses it directly.

So far, this Go server is turning out great, and I’m excited by the fact that its compiled and faster. I don’t have to distribute my source code now :-)

Performance

* Disclaimer: These numbers are just comparisons between what I built in python vs Go. I think the point is to reflect what I naturally came up with on my first pass at using Go, vs applying years of python experience. I’m dead sure my Go code isn’t written as efficiently as someone with more experience, which I think makes it even more interesting of a comparison.

Go comes with built in testing and benchmarking functionality. What I built was a client test that connects to a running server and rapid fires messages. It times how long it takes for a 150 byte message to be sent out, flow through the server, and come back to that client as been delivered. The gotest utility that is used to run the test code will run the test, and if it ran too fast to calculate timings, it will repeat the test over and over with larger iterations. When I first got my Go server working to where a client would send a message and it would just get broadcasted right back out to everyone, I ran a benchmark. Here are the results of my tests…

TEST TIME PER MSG NET RESULT
Python (Tornado, …) 835314 ns/op -
Go (barebones messaging) 107091 ns/op 7.8x faster
Go (1-to-1 python port) 159823 ns/op 5.2x faster
Go (weekly.12-01-2011) 76230 ns/op 11x faster

 

The second Go test was after I finished implementing the same 1-to-1 feature set of the python version. I had thought the numbers would be dramatically impacted after adding the overhead of all the internal message routing, but the Go server still came out almost 5x faster than the python version. And this is from my first attempt at writing a Go program! I bet once I get a lot more solid with the language I can optimize this code a lot more.

Conclusion

I’m pretty hooked on the language. I feel its the perfect option for a python programmer that wants some speed increases and simple concurrency, without having to learn something as intense as C++. Go is supposed to get faster and faster as they improve things like the goroutines, channels, and the garbage collector, so its a great time to jump in and start learning. Its really helped me understand more formal concepts that will probably make learning C++ even easier once I decide to go back to learning it :-)