devmorgan.com

Recently I wrote a few new algorithms for a college blogging site that I am apart of called The College Blog Network. I went to lunch and had a few beers with Dan (the lead developer), and basically discussed with him the good and the bad of TCBN. The site is a great idea, and has a lot of potential, but we both agreed the scoring algorithms needed some work.

So, with a little discussion, I had come up with a couple algorithms. One of which has been implemented already, and one that will be implemented in the near future. I have outlined the general workings of these algorithms, but left out some of the details to prevent unforeseen exploitation.

Hot Topics Algorithm

The main goal of this algorithm is to score a blog entry based on how “hot” it is. Hotness is essentially popularity and freshness. Even if a posting was one of the most popular, 3 months later, it is no longer considered hot. At the same time, a post that has been popular for the last week should still be considered hot, even if it has been around for a few days.

The solution was simple, apply a weighting system to the score that is used to calculate “hotness”. The main factor is time or “freshness value”, which is an adjustable value that can be changed to accommodate TCBN as it grows.

As shown in the graph, full weighting will be applied to newer posts, but over time, the weight a post has will begin to degrade. Lets say your initial weighted score on your post is 10 initially. After X time, it will degrade to 5. To maintain a weighted score of 10, you will need a unweighted score of 20. This will allow for consistently popular posts to remain hot, but only initially popular posts to fade away quickly. Thus maintaining an accurate “hot list”.

Overall Blog Scoring Algorithm

The second algorithm I worked on is much more complex. The goal of this algorithm is to try to calculate the “goodness” of a blog. Currently the overall blog score is calculated by adding all blog “bumps” together, then removing the blog “dumps”. In the case of a tie, the blog “views” are used. While providing an decent ballpark estimate, this score is not easily changeable (you must specifically go out of your way to bump a blog), does not account for postings, and can be exploited positively and negatively if a group of people bump or dump all at once.

The main goal of this algorithm is provide accurate ratings for blogs. The requirements were that a visible score would still need to be generated, it would still be primarily based on user feedback (bumps/dumps), and would be less susceptible to exploitation.

Simply put, the new algorithm breaks down to this:

Base “Goodness” + Posting “Goodness” + Viewability

I can’t reveal all the details of how each value is calculated, but here is the gist of it:

Base Goodness is essentially the current rating system, with some minor changes to make it a little more robust.

Posting Goodness takes consideration of posts scores (which will be more apparent when post bumping and dumping is added in the next TCBN revision). This value considers all posts. If one post in a great while is “good”, your score will not be as high as if most posts are considered “good”.

Viewability is a non-user controlled value that takes unique post views into account. A post that never gets bumped, but is viewed a lot will contribute a small amount to the overall score of the blog.

Overall these algorithms were a fun little side project for me, and hopefully will come in great use for TCBN. Tweaking might occur once we start to see how the scores turn out, but these are the basic principles behind them.

Alright, here comes a rant. Sorry about this, but tutoring a student in PHP is painful. In fact, it’s not just PHP, but I am using PHP as an example.

Don’t get me wrong, I love PHP, but there is a single reason why tutoring people in it is so hard. The people.

People can need help with various technologies, and that’s fine. In fact, I really adore helping someone that knows how to program with a small problem. The problem with PHP is, most of the people that require assistance, don’t know how to program.

Setting people up for failure

PHP is used in the multimedia / web realm here at RIT. Sure, it’s used in database as well, but I rarely get asked for assistance from database students on how to use PHP, and when I do, it is more of a technical issue. The problem is, most students that focus in multimedia / web are not programmers. In the Information Technology program at RIT, you are required to take 3 courses in Java. Once you complete these courses, it is relatively easy to avoid programming in most of your courses for the rest of your tenure here. Students that did poorly in Java tend to go towards multimedia / web because it is easy. You can get away with almost no programming (writing HTML is not programming), and you are not forced to push yourself constantly when it comes to programming.

Enter PHP. As students hit the mid level courses, after trying to avoid programming for a long time, they have a new paradigm, stateless programming. Each time you post to a page, you lose all the data from the last page (unless you pass it along through POST or GET). Students struggle. Constantly, I have to work through basic programming concepts, and these frustrations are made worse by the lack of understanding of the state problem. Combine these weaknesses with a grasp of simple programming with the introduction of a database (which happens in some mid-level courses) only having phpMyAdmin as a guide, and students begin to drop like flies.

What is the problem?

Students pass the initial Java courses, and they are expected to be programmers. Unfortunately they aren’t. In fact, I know first hand that many students that finish the last course in Java couldn’t program anything remotely complicated if they were required to. Beyond this, with a weak foundation, they can not easily grasp another language. They never learned what makes a good programmer, and spend most of their time shotgunning their way through. Professors in later courses never cover the foundations again, and students get lost.

How do we solve it?

I suppose if I had a say, the Information Technology department would be much smaller. What it really comes down to is, students should be flunked in the early Java classes if they are not solid programmers. Yes, it’s a mean thing to say, but if you can not be solid at the core concepts, you are going to be struggling for the rest of your career. Programming is a major part of Information Technology. I will make a claim now that goes against what many of the “higher ups” in the department think, and what they would hate for me to say…

If you can not program, you will not be successful.

I admit, there are likely some people that couldn’t program their way out of a paper bag that are successful in the IT industry, but this is rare, and partly dependent on the definition of ’success’. Someone that gets a job isn’t necessarily successful (my definition may differ from others on this point). However, someone that can be innovative and inventive in the industry is what I would consider to be successful, and frankly, I can not see someone that can meet my criteria for success in IT without being at least a ‘decent’ programmer.

Whole rant aside, I want to make it clear that just because someone is in the IT program here at RIT, and focusing in multimedia / web, it doesn’t make them a slacker or bad at programming, because the concentration is, as with life, what you make of it.

For the last several months, I have been a programming and database tutor here at RIT in the Information Technology department. I have seen many students come and go for help, with varying levels of skill and motivation when it comes to programming.

I hate to say it, but I usually find my self disappointed in some students. I have been trying to figure out why they struggle with programming so much, and I want to try to help them get past the roadblock they are having… I have determined that one of the biggest roadblocks that students run into is having a profound lack of attention to detail and no patience.

Many times I get asked a question that could really be answered if someone just took some time to look at the code in detail, or think through a concept fully. I believe that people are fully capable of logical thought, so just taking some time is all that is really needed.

Step 1: Determine the problem you are trying to solve.

Even at the low academic level, you are programming for a reason. Before being able to code anything of value, you need to be able to abstract the fundamental essence of what it is you are trying to accomplish. If you are writing a program that will calculate area of shapes, that is one paradigm, another is connecting to a database. You must tune your brain to the frequency of the problem, and attack it from there.

Step 2: Develop a plan

Over the past few years, I have determined, I am not one to spend lots of time on design. In fact, I find that most design methodologies are too much overhead or too optimistic. I do however believe that you must always have a plan for what you are doing. A complex UML diagram is not needed, but a simple “WTF Napkin-gram” should usually suffice. Sure, part of this post is about detail, but the detail is most important in the code, not in the plan (this is my opinion… methodologists will wholeheartedly disagree).

Step 3: Code, and follow the plan

Here is where things start to go awry, and this is where I want to focus on a few key problems that I run in to.

People tend to write code blindly. You need to think about what you are writing. Every line should have purpose, and should be directly trying to accomplish the goals in your plan. Every line of code that you write that deviates from that plan is code that is considered ‘junk’. Look at the documentation for your language, and make sure you are not reinventing the wheel, but more importantly, understand what each function call you make does. Does it return a specialized object that behaves in a certain way? What are the bounds of your parameters? Take the time you need to make sure you understand each piece of what you write.

Focus on the error messages. A lot of the problems that I get asked to help with are (relatively) simple compile/syntax errors. Every language has some way that it can get these error messages to you, use them! This is where your attention to detail really comes in handy. You can look at a block of code and see things you might have missed, incomplete statements, and most common (but not always an error) incorrectly named variables. Make sure you understand the code you are touching, and be very explicit in fixing problems with it. Don’t shotgun debug, be precise about what you are trying to do,

Copying code is a bad idea… usually. This is one I am guilty of, in fact, most programmers are. There are several major flaws you need to worry about when you notice yourself copying code. The first problem is the most obvious, if you can copy code, there is a chance that the branch of code you are copying should be in a reusable function. Look at the code, see if you can abstract it to be a function call, and if so, do it! The second major problem with copying code is that its extremely easy to miss any changes you need to make to the code. For example, if you copy a few lines that are modifying variable xmlDoc, and you want the copied code to modify xslDoc, you might miss a change in going from xmlDoc to xslDoc. Happens all the time… so be careful. The third major point I want to touch on with copying code is this: I don’t recommend copying more then 3-5 lines of code at a time from another project. Be sure that you actually need the code you are copying. If you just grab an entire class file, you will likely need to change it enough that it might be a safer bet to just rewrite it… of course on that note…

Don’t reinvent the wheel. If code exists that does what you need, use it. In academia it is a little different, since we are trying to build up core concepts, but, in reality, if it has been done, you likely do not need to do it again. Many people could go on for hours about how important it is that programmers be lazy. I agree, programmers need to be lazy so they can see the way they can save time while coding, but remember that programmers also need to be proactive. You need to be able to spend time looking at documentation, and just playing with existing code to see what is already out there. Save yourself a headache later by taking some time to discover things now.

Program for programming’s sake. I guess this will be my last big point. A lot of people only code because they have to. Sure, this stuff isn’t for everyone, but if you want to be a good programmer, you need to do it a lot. Practice is how you begin to learn all the documentation, and all the tricks about a certain language (To Students: Yes, that is why I know where everything is in the javadocs). Really, the best thing I can recommend is to come up with a pet project, and do it. If it suits, trash the code and write it from scratch after you work out the initial kinks and know the problems you will run in to. The more you code, the better you will become.

These are only a FEW tips for becoming a good programmer. They are just some of the big issues that I see on a day to day basis with students here.

This past quarter, I was taking a class in which I got to play with web services. I developed a web service in Java and used the latest application server, Glassfish, to deploy it. To consume the service, I was going to write a C# desktop application, and a Flash Lite mobile app. I thought it would be very exciting, and a lot of fun.

I was wrong.

The C# app was nothing remarkable. In fact, I was very impressed at how easy Microsoft has made web services. Just point Visual Studio at the auto generated (more on this later) WSDL file, and you can make the calls easily.

The flash application was…. more of a problem. So, in an attempt to explain why it was a problem, let me make a nice broad claim…

Flash will not work with Glassfish.

Random Yokel: But Dave! Isn’t the point of web services interoperability?
Dave: Why yes! Which is why I don’t know who to be more mad at…. Adobe or Sun!

So, Lets discuss what is wrong!

Using Glassfish, you automatically generate a WSDL based off of your Java code. The WSDL itself is valid XML, and a valid WSDL. However, when I pointed my little flash mobile test application at the WSDL, it failed. The error that Flash Lite 3.0 gave me was:

There are no valid services/ports in the WSDL file!

Weird. Especially because there definitely is a single, valid service and port in the WSDL… but that’s ok. I tried to turn it into a standard Flash application (rather than a Flash Lite 3.0). I removed the little bit of ‘mobile only’ code, and ran it again as a Flash 9 app (but the code was still ActionScript 2.0). It loads… so I try to execute my helloWorld() method (just returns the String “Hello World!”), and it fails:

Element tns:helloWorldResponse not resolvable

Strange. However, at least it appeared to get farther this time. Now, this error message gave me a little bit of a headache for a while… and then I found out the Flash does not support xsd:import. So, because the WSDL file that Glassfish will automatically generate (I suppose I should have guessed it would cause me problems), includes an import to an external schema defining your data types, Flash will choke. Ok. Let’s get around that. So, I took the WSDL that was generated by Glassfish and merged in the schema file. Simple enough, no problems. Run the code, pointing to the new WSDL which I hosted on my server (so: http://localhost/newWSDL.xml, rather then Glassfish app server: http://localhost:8080/Path/File?wsdl)… New error:

Unable to connect to endpoint: <PortURL>

Damn it. Nothing appears to be wrong with my code, C# and PHP can easily connect to my service. I was able to find this document, which basically says that Axis (another Sun app server) is sending poorly formed SOAP packets, and the solution is purchasing Flash Remoting. Right. I believe that. If the packets were poorly formed, I should be getting errors in other languages. Besides, my helloWorld() method is only returning a String! Personally, I want to think that it is some Flash sand-boxing issue, but I truly can not figure it out. Why this is so difficult? My final conclusion on this project was that Flash can not connect to a Glassfish app server (at least, out of the box). If someone has figured out how to make the two play nice, I would love to hear it.

This is bad news for Sun and Adobe. The point of web services is interoperability between systems. If you can’t deploy a simple application using Glassfish (which is the new Sun App Server…) that can be consumed by a simple Flash app… what is the point? Sure. I could go the Microsoft route, but what if I don’t want to? What if I can’t?

So, some people get all hyped up about Valentines day… other people don’t.

This comic is for the database geek that got all excited and swept up, and then realized the truth. Yes, I used most of the art for the last one, but give me a break! Finals are coming up!

Sometimes it just isn’t there. If it’s only there around February 14th… maybe it isn’t real :-P

In anticipation of Valentines Day. I figured I would draw a comic, where a geek professes his love.

In summation:

for (int i=0; i<ways.length; i++) {
   print("I love thee, " + ways[i]);
}

Have a great Valentines day!

I have been going to RIT for a while. Grad school is still fairly new to me, and it really has been the first time I have been challenged. The work isn’t particularly hard, but the most difficult part is the intense amount of work expected. Throw in the 20 hours of tutoring and teaching I do, and I start running out of time fast.

What does this mean? It means that I spent last night’s exciting superbowl in my living room with my laptop coding (although, to be fair, I was pretty much only paying attention to football by the 4th quarter). I was hoping to draw a comic last night after the game, but since I was experiencing a delightful bug with glassfish, I was unable to get to it. This is a direct side effect of “crunch time”.

Weekends are assumed to be spent in the labs now, be it for tutoring or to try to get my projects done. I knew this coming in, but it starts to get worse and worse as the quarter ends. Since RIT has 10 week quarters, when the crunch starts to come, the best students are the ones that are adept at the art of triage. Which assignment is worth most? Which assignment has the closest due date? How much will not doing this assignment hurt me? These are not good questions to have to ask, but in a student-academic setting it happens a lot.

(several paragraph rant on homework assignments deleted, perhaps I will write an post on it another day)

So, it is crunch time here at RIT. You can see it in the eyes of the students, you can start to hear the sense of urgency in their voices. General grumpiness levels are up due to lack of sleep, and labs are starting to get more and more crowded as the final weeks tick away. Not much can be done to prevent the inevitable, but some thing can be done to keep yourself sane.

1. Set aside 30-60 minutes a day to have fun
2. Have one fun thing to do a week that takes up a little bit of time (usually Friday or Saturday night)
3. Smile, at least a little.

Making sure that you get to at least spend a little bit of time a day to do something fun (be it playing pool, playing a video game, watching TV, or hanging out with a friend for a meal) is vital to keep yourself from burning out, and burn out is the worst thing that can happen here. Shutting down at the worst possible moment, the last week of the quarter, is what needs to be avoided at all costs. So, grab that cup of coffee with a friend, let your brain reset before getting back to work.

Do at least one big thing that is fun a week. If it is going to a party on Friday night, or maybe going out bowling with friends, it is what you have to use to drive yourself to make it through the week. Think of it as the reward for accomplishing the work you have been doing all week. Make sure you don’t get too crazy, because I know programming with a hangover sucks, but make sure you relax.

Smile. Please Smile. This is the toughest one for me. Despite the scientific proof that smiling is actually good for you, this is just as much about keeping everyone around you from getting more and more down. Seeing someone smile is… rare… here at RIT. Try to make the last few weeks of the quarter as painless as possible for yourself and everyone around you - if you smile, it adds that little bit of happiness to the environment you are in… and if enough people do it, maybe the end of the quarter will just be busy and stressful, but not depressing and painful.

One of the most painful things I experience with the students I tutor is the practice of Shotgun Debugging. The practice will generally give the student horrible, unreadable code. After today’s tutoring, I determined it was time to write about some common debugging practices. So, here are the things I tell my students every week - and they quickly forget.

Only change one thing at a time.

This one is hard for beginning programmers. They get a logic error (or sometimes even a compile error) and then change 5-6 things in their code before they try again. If the changes work, they have no idea why, and will change 5-6 things next time they get a similar error. Which leads me to the next point:

Learn what the common errors mean.

….and how to fix them. When you get the error in Java for “cannot find symbol”, what does that really mean? Look at the text associated with it! It will tell you if it doesn’t understand the class you are looking for, or the variable, or even the method. Got a “null pointer”? That really does mean that something you are referencing is null! The JVM does not lie! Sure, some errors are a little more complicated, for those:

Actually read the error.

The biggest problem I see with students is they simply do not understand the code that they are working with. Reading the errors and understanding what is actually happening when you manipulate a line or two of code is the first step in becoming a better programmer. Every good programmer has spent a significant amount of their time learning how to debug, not just how to code. Lastly, I find myself telling my students:

Make sure you catch every instance of the error.

Let’s face it. We try to teach reusable, generic code, but it just doesn’t happen. I have learned to accept that some students are just going to copy and paste branches of code, over, and over, and over again. So, if you must do this, when you realize there is a logic error in the branch you copied, it will exist in every branch… no matter how much you think you “just fixed it”.

Sometimes tutoring can be very frustrating, but I really do enjoy the moment when a student finally has it click. I suppose there is nothing I can do sometimes, because some students just really hate code, but I hope that they can at least take a little bit back from what I help them with.

I think I might file this one under “Bad jokes I should have never drawn”, but I still get a chuckle out of it.

I think this is a sign that I need to stop coding for a few days…

Something interesting happened tonight, between the hours of 3:00am and 4:00am (can you tell I haven’t been getting sleep?). The interesting thing that happened is that I trying to find plug-ins for WordPress that would allow easy sharing of articles that I write with facebook, digg, slashdot, etc…

I found one that seemed promising called obsocialbookmarker. He had a new release out, so I figured I would give it a whirl and see if I liked it… because I really didn’t feel like writing something custom that I knew someone out there had already done.

It didn’t work!

I was frustrated to say the least, but I was able to at least get some of the error messages to display to me. The errors were not WordPress errors, but actually PHP errors from the plug-in code. I thought “Damnit this guy must suck”. Then I realized what the error actually meant… The plug-in tries to figure out if it is out of date. Fair enough. The problem is that my server does not take to kindly to PHP trying to call fopen() on a remote server via URL (a security setting I cannot change, since allow_url_fopen is disabled… although this does give me more motivation to try to get around that).

So what was I to do? I suppose anyone that wasn’t a programmer wouldn’t have even got this far and would have moved on to another plug-in, but I saw some of the other ones and I thought they sucked. So, I did was most programmers like to do, and hack. Sure, the hack was only removal of the version checker, but it was still something kind of neat.

1. Something didn’t work.
2. I wanted it to.
3. I made it work.
4. I posted a comment on his blog to let him know.

Sure. This isn’t anything huge, and I have been a programmer long enough to know that this happens all the time! I think tonight something was just different about it. When I was learning how to code, I used to talk about how being a programmer was akin to being god-like. It kind of motivated me to learn. You can create things, destroy things, do good, do evil. This kinda just touched me again to think about that concept… what kind of power a programmer has.. Sure, for me just now, it is as simple as making something work that will put little icons on every post I make. The bigger thought perhaps, is what makes the idea interesting.