devmorgan.com

So, before I sit back and talk about all of the actual methods on the MagicDB object, we need to talk about a problem that is specific to PHP and the mysqli extension.

Now, the basic idea I had was to build something that could generate prepared statements dynamically. This will (if done correctly) help protect against SQL injection, and ideally help speed up a few frequently run queries (as long as the SQL structure is always consistent). We need to select mysqli because it has the mysqli_stmt::prepare ( string $query ) function.

Ok, so, now we come to the binding parts of the process… and here is where it gets sticky. We are presented with these two methods:

mysqli_stmt::bind_param ( string $types , mixed &$var1 [, mixed &$... ] )
mysqli_stmt::bind_result ( mixed &$var1 [, mixed &$... ] )

This presents us with a problem. Notice how both methods (which are required for prepared statements) allow for multiple parameters. This is handy — wait.. really.. it’s never handy. I really don’t know why they designed it this way. In fact.. Damn those PHP people. OK.. so maybe I do know why,  but it gets in to reference/value discussion.. so.. lets move on and just figure out a way to fix it.

In an ideal world, for being dynamic, the functions should be designed something more like this:

mysqli_stmt::bind_param ( string $types ,  array &$params )
mysqli_stmt::bind_result (  array &$results )

Instantly this becomes a lot better. Since the parameters/results are stored as an array, they can be dynamically generated and of any length.  To address this concern, we need to tackle both functions a little bit differently. First off, the bind_param function:

I wrote a simple private function that is included in MagicDB that will accept a mysqli_stmt object, and a list of parameters. From it, the function will call the bind_param method with the proper values and return the statement object.

Now, lets discuss the major concern with this code. The data types of the parameter array that is passed in are used for truth. So, as a developer you need to be careful. This will not cause any security risks, but it may reject your parameter if the wrong data type is used. This is a known concern, but one that I am not worried about since I know to pass an int when I need an int. The beauty of this code is in the last couple lines. I use the php variable-variable-name functionality to generate a dynamic variable reference. I call each variable $bind#, and then assign the parameter to it. I then can use the & symbol to add the variable by reference to the array. Using  the PHP function call_user_func_array, we can get around the multiple parameter problem by simple passing an object reference, a method name, and an array of parameters.

Now for the binding of the result. This code isn’t stored in a function, although it could be. It’s stored in my “get” method on the MagicDB object. Some variables are out of scope, but here is the snip.

So here, without getting into too much detail about how the get() function works, we want to return an object of the type that $object is.  The object should have all the values populated with the results of the database. We bind the variables in the same manner that we bound the parameters in the previous function.  Where it gets fancy is in the use of reflection. We get the type of the object, and pass it into what is called a ReflectionClass.

This will generate a generic class of the type we specify. We must then instantiate the object by calling newInstanceArgs() method. This allows us to pass an array with the parameters for the constructor. This is why the constructor is required in all of the data objects. We do this for each iteration of the fetch, and return the array of objects.

Sexy eh?

Next up we talk about the API and how the MagicDB functions actually work.

dave coding, data, technology

MagicDB is the codename for the Dynamic Data Layer that I built to help me with my much larger project, my fabled “Capstone” for my Masters of Science Degree. First a couple technical details:

• MagicDB currently runs on PHP 5.x.
• MagicDB currently only interfaces with MySQL.
• MagicDB is built using an Object-Oriented paradigm.
• MagicDB uses PHP’s mysqli engine.

Understanding the technical bits will explain why some decisions have to be made. PHP is used pragmatically for myself, since that is the language I used to build my capstone with. It’s free. That helps. MySQL is also free, and one of the most common companions with PHP. The mysqli engine is used primarily because it provides the support for prepared statements. Using a prepared statement allows us to curb SQL injection attacks, which addresses one of our main concerns with security.

MagicDB has only a couple rules that need to be followed.

1. Data Objects (PHP) must follow specific naming requirements.
2. Data Objects (PHP) and Data Tables (MySQL) must follow specific structural requirements.

Naming Requirements are actually pretty simple. All data objects used with MagicDB need to follow the pattern [Prefix]TableName. The prefix can be whatever the developer wants. For my capstone project (codename: TWiG), I used the prefix “TWiGdb_”. This means for a table in my database named “Entry”, I have a corresponding data object with the class name of TWiGdb_Entry.

Structural Requirements are a little more in-depth. Originally, the structures were required to be identical. This meant that for every column in the database, the data object should have a column for that as well. I improved upon this by adding a column to every database table (but not the data object) called “exist”. This column is a flag to show if the row was removed from the database, it has a default value of true. This allows us to “remove” data, without deleting it (although functionality is built to actually DELETE data), in case we require that data later.

Another structural requirement is that the first column in the data object MUST be the primary key for the table. In the table, the first column must be the primary key as well. This allows us to issue UPDATE commands easily. A constructor with all the attributes as parameters is also required. Order of the parameters and attributes must also match across the board.

The final requirement is that each attribute of the data object MUST be using the same exact name as the corresponding column in the table.

For example the table User would be described like this:

The corresponding data object would look like this:
Once we can build the structure of the database, and the corresponding data objects, we can begin to discuss how the DyDL works. MagicDB itself really has a simple interface to use.

In the next few posts we’ll discuss each of these methods, and talk about some of the private methods that power the implementation.

dave coding, data, technology

So, I figure it’s about time I actually talk a little bit about code. I know many of my readers aren’t programmers, so my apologies. For my capstone work I have been playing around a lot with data, and I want to share my experiences.

So, first things first. Lets talk about Dynamic Database Layers. We want to build a database layer that is as dynamic as possible and flexible, but provides understandable structure and expected results. We want something that is secure, but also usable. So lets break it down:

What are the goals?

1. Ease of use.
2. Low learning curve.
3. Clear and understandable “rules”
4. Secure
5. Solves routine problems naturally.
6. Allows non-routine problems to be solved easily.

Ease of use. The first and foremost goal we have for our DyDL is that its easily usable. Ideally, it should make accessing the data simple and trivial. If you need to write lines upon lines of code just to invoke simple calls, developers won’t use it. Make easy things easy. If a developer needs to do something more complicated, try to make it easy - but they’ll be willing to do more work to accomplish it.

Low learning curve. One of the most difficult tasks as a programmer is to adapt to new technology. Understanding the lingo, the structure, and the benefits is required to be efficient. Don’t make it difficult to use. Use common paradigms to make entry into use easy. This leads us into the next requirement..

Clear and understandable “rules”. Make how the software layer works straight forward and simple. Don’t build an application that has situations where if you want to do one thing, you have to do it differently than another thing. Make it clear to the developer what each function is actually doing. If all data objects require certain structure, make it clear. Giving meaningful messages and documentation for the layer.

Secure. This is straight forward. Make your best attempt to make the code secure. If you can prevent SQL injections, do so. Build as much structure to the framework such that you can limit the ways to exploit it. Don’t trust unknown data.

Solves routine problems naturally. Make simple stuff simple. If you need to do a simple access of data based on a few parameters, make it so that is one simple call. Try to abstract your calls so the developer doesn’t need to know the SQL that is being run underneath. Allow simple complexity by allowing the developer to do so, but don’t require them to.

Allows non-routine problems to be solved easily. Always allow the developer to decide if their way is better. Provide a simple interface such that a developer can still use your engine to do things that you haven’t accounted for. If you can, try to make it easier. If you can’t, don’t stand in their way.

For a portion of my capstone, I have implemented what I believe is an excellent start of a DyDL, codenamed MagicDB.  I will be writing about this project more in the coming weeks, and will be showing bits of code, and explaining decisions made. Hope I don’t bore anyone.

dave coding, data, technology

I have heard this word a lot lately. It partially explains why my writing has been more sporatic lately. So, for that, my apologies.

So. A few big things have happened in my life, some of which geekier than others. First things first: I’m engaged! This is very exciting, and well, pretty much awesome. For the record, I didn’t do anything too fancy, but I did do the one knee thing.

Now on to the geeky things! I have been working on a research project for about a year now, involving Data Mining and Data Warehousing. Dr. Jai Kang, his son, and I have written a paper based on the research, and it was accepted to a conference! In Italy! So, come May, I will be heading out to Milan to present the paper with Dr. Kang. Very exciting time indeed. I will likely make a copy of the paper available here after it is presented.

I am also working on my capstone. I am staying pretty quiet about it online right now, but I can tell you it will be really cool. It will involve a visualization, and will be available as a web application, and of course, I will allow access it from here.

I also have had several interviews, hopefully I’ll be getting a nice development job in the coming months.

I guess the long and the short of it is, I will be posting more soon I hope. I just need to catch up with everything that is going on in my life. So, until then, adios.

dave life, relationships, travel

Another call from the parents, wanting to get something working correctly on their computer.

You’d think we’d be past this… but we aren’t. That’s the problem with technology today. We are SO close to having things “just work”. In fact, I’ll admit, Apple is closer. The problem is, we aren’t there yet, even for the mac.

My parents are smart people. They don’t understand computers, but they are smart people. In fact, my father is one of the most intelligent people I know. He has a mechanical mind like none other. Once we start talking how to use the computer though, things get increasingly difficult. I wish I didn’t have such a short fuse, but I get frustrated easily with technical questions. Sorry guys. The problem still remains, when things get plugged in, and installed… they don’t always just work.

The reality is, will we ever get there? Sure, things are more and more user friendly, but sometimes the learning curve is just going to be difficult. Trying to explain how to get video conferencing is hard when the person who is being told is miles away and doesn’t understand why the computer needs to reboot to recognize the drivers, or where the unmute settings are.

This isn’t meant to be a rant - wait - I guess it is, but as software developers we always need to strive to make things “just work”. A lot of times, this isn’t the focus. Sometimes hardware is the limitation. Sometimes software is.  Sometimes the focus is quick development, and not quick learning curve. Sometimes we just specify a “requirement” on the system to avoid something that will cause the program to break. Although, every day we get closer!

What I worry about is that more and more, I need to explain to many of my students that things are “magic” and it just works. This is great - but it’s also scary. We need to be able to develop things that “just work”, but at the same time have a very clear and logical paradigm that is associated with them, so they can be understood. I don’t know how to solve this problem. Hell, I can barely think of examples. But, it’s been a month since I’ve written and I figured I’d say something.

Perhaps a few more days in the oven and this thought will be a little more cooked.

dave coding, technology

Sure. I didn’t get this post up before Christmas. Big deal =P.

Happy Holidays to everyone. Right now I am spending some time in the Philadelphia airport waiting to catch my flight to head to Cincinnati. After that I’ll be heading to Boston for new years. I got a busy week ahead of me.

The point of this post is simple, last night I was asked to say grace at dinner for my family. What I said was simple: Lets be thankful for this past year. I did not mention any God, I did not mention any faith. I simply asked my family to reflect back on the past year and be thankful and proud of everything.

My family has two new members this year, one comes in the form of a canine that my dad loves to death, and the other is my nephew Cole. We have a lot to be happy about over the past year, and a lot going for us. I just want to remind everyone to take some time during this holiday season to think about the year and be thankful for what has gone well.

Merry Christmas, Happy New Year.

dave life, travel

Several months ago, there was an XKCD comic called Morning Routine. It is the original inspiration for this comic. I started to think about when I end up sleeping with my laptop in bed. It’s when I’m alone, and usually only when I am feeling pretty lonely.

It’s kind of a painful reminder, isn’t it?

dave comic, life, relationships

So, I’ve been gone for a little while. In fact, too long.

I’ve had a lot going on in my life, and I find it harder and harder to get myself to sit down and write. My appologies to anyone who reads this blog. However, I have something to show for it.

The past several weeks, my roommates and I have been putting a lot of our effort into building our new bar.

Kevin is quite the carpenter, and did much of the framing and much of the bar. Justin and I were able to help out also, particularly in the finishing work. Overall the bar’s dimensions are pretty big. It is an L shape, and comes out from the wall about six feet, and then tails off for another six. We designed the bar from scratch and built it as such. There is a draft beer system, consisting of a refrigerator conversion. We can store a quarter keg of beer on tap. We hope to finish the entire basement (where the bar resides) and fit it with a projection system. Should be great for the rest of football season!

PS. I have a new comic idea, should have one out soon.

dave booze

Something bad happened a few weeks ago. My desktop died. But, here is the surprising part:

I haven’t replaced it yet.

Yep. Dave Morgan, hyperconnected geek extraordinaire does not have a permanent presence online. Not gunna lie, at first it was scary. I had my addiction, and I thought for sure my life would be less fulfilling if I lacked my technology.

I was wrong. Sort of. I adapted.

The truth is, my desktop really didn’t serve much of a purpose. I called it my “social box”. It was a machine that was constantly on, was always running Pidgin, twhirl and my e-mail clients. I don’t need my desktop to get my work done.

Of course, I have seen use of my iPod touch go up at least five-fold. The touch has become my new “social box”. If I am anywhere near a wifi point I am constantly checking twitter and my e-mail. My laptop has become the new home of the instant messaging, but I occasionally use the touch for that as well.

So, am I less connected? No. Sure, my day to day life has changed a little bit. I lack the “always on” connection that my desktop provided, but at the same time I now have developed a deeper mobility in my tech life. I have adapted such that I no longer need to be sitting at my desk to do any work. Period. Most of my “play” tasks get accomplished on my iPod touch, and serious work will be done on my laptop.

I don’t think I’ll be in a rush to replace my desktop. Eventually I will, but, for now I don’t mind. When I do replace it, I will likely give it a new purpose (I have been toying with the idea of using it more like a media PC). The mobility is refreshing. I am disconnected physically from the wall, but I don’t think I’ll ever be disconnected from the internet.

Besides, now I can claim I am doing my part to save the planet by consuming less energy. Yep. I’ve gone green.

dave technology

I want to let you in on a little secret: Technology doesn’t solve every problem.

In fact, it sometimes creates some.

My friend Elvis wrote an article about a conversation we had a few weeks ago. We were talking about communication mediums. The article he wrote is a much shorter distillation of what we discussed, leaving out key points (editors and word counts! Ha. Who needs ‘em!). So I wanted to touch on some of the discussion as well.

The first thing we have to talk about is hyperconnection. In reality, we have a pretty sweet deal right now with technology. If we want to get in touch with someone, or send someone a message… we can. Send a text message or an instant message. Done. Want to set up plans for tomorrow night? Done. What an excellent ability to have! There is only one problem, it’s a horrible method of communication.

In fact, I text and IM constantly. We don’t even need to get started about how much I use Instant Messenger and GTalk, but on any given day, I’ll send and receive over 40 text messages. For some, that’s a lot, for others, that’s nothing. To each their own. However, even though many communication tasks in a given day are similar to the ones I just mentioned and they are perfectly fine remaining asynchronous. Others are not. Others require synchronous communication.

Asynchronous communication isn’t all it’s cracked up to be. Sure, it’s ok to shoot off a message and not have to care about the response being timely. In fact, a lot of times you don’t even need to worry about if the message was received at the same time you sent it. “Hey - see you tonight at 8″ isn’t a problem. The problem is when you use an asynchronous communication method when the communication clearly needs to be synchronous.

I suppose we should go over what the difference is:

Asynchronous communication includes text messaging, emails and instant messaging.

Synchronous communication includes telephone calls, video chat and meeting in person.

Do you see the difference? The first set lacks what is most important in communication: an implicit expectation of immediate response. When you stick to conversations that can be done asynchronously, it doesn’t matter at all. However, if a conversation turns to something that requires responses and is actually meaningful you must be wary of the asynchronous methods. Nothing is more frustrating than trying to hold a conversation with someone that isn’t paying attention. Using the asynchronous methods that is exactly what you are inviting. The reason they are so damned convenient is because you can do them while doing something else. This is only the start of your problems though, because unfortunately the mediums that are asynchronous also lack other required communication features.

I don’t know about you, but my thoughts can not be expressed in 160 characters. Text messages are short. Sure, you can link messages to send a longer one, but, in reality, you are typing on a keyboard smaller than your hand - you aren’t going to be saying much. The same can even be said for instant messaging and emails. Once you turn to these mediums, the accepted practice is to put as much information into text as humanly possible. Again, for some communications this is perfectly fine. You can say exactly what you mean by using a few words. However, if you want to have an important, meaningful conversation with someone, you will be out of luck.

Written word is one of those tricky things. Determining tone and meaning from a particular piece of text can be very difficult. In fact, its very easy to misread a particular message and interpret a very different meaning.

For example, one of my most hated words in English (but, actually one of my favorite in ASL): Fine.

What does “fine” mean? Just the way it sounds in your head can be different.  When reading the word, it’s  tone is ambiguous. You need to take other cues to try to understand the meaning behind it. Does the person mean “everything is fine” or “things are… fine“. Responses to requests can be met with “…fine.” which has a much more negative connotation than “fine.”, but is still not as great as “fine :)”. Things can be “fine with me.” or “fine with me!” and they have a different meaning. All of these different versions need some context, some clue to help explain what they are really meant to mean. This is where textual mediums fail. Sure, I’ll use over my fair share of emoticons to try to express meaning, but you just can’t get the same effect as using verbal and visual cues from conversations when using the telephone, video chat or actually physically standing to chat with someone.

So, we have new communication mediums that are great in certain situations. Sending short messages, or trying to get a message to someone that may be busy has become easier. However, with these new mediums, it becomes more and more the norm to attempt to use them for what they can’t do: real, deep, valuable conversation.  You simply can’t depend on a text message to consult a friend about life decisions. Just as much as you can’t call someone 3 times an hour just to tell them something random that just happened. There is a time and a place for each medium. The really important part is deciding which to use when.

dave life, rambling, technology