Neo and Smith are the opposite sides of the same unbalanced equation. What the Machines are trying to do in The Matrix is to create a equation that is equal on both sides: X = Y
Making sure that X = Y is The Architect's purpose (remember that while he might be sentient, he does have a directive). The Architect's job is impossible due to an unforseen and incalculable variable: free will. Free will upsets the balance and: X != Y
The Architect, following his directive, inserts a variable to compensate and we get: X + Neo = Y
However, Neo himself has free will and continues to grow in power - once again upsetting the balance. The Architect does his job and inserts the opposite: X + Neo = Y + Smith. Put another way: Neo - Smith = Y - X
This shows us that Neo and Smith are the opposite variables in a balanced equation (which The Architect explains in serious detail if you can make heads and tails of it all). "Neo wasn't in the first version of The Matrix," you say - no, his variable was and The Architect balanced him in the same way (remember machines don't care about people, they care about variables). This is why Neo fights literally to the death because he realizes that in order to remove Smith from the equation, he must also be removed from the equation. His (and thusly Smiths') death restores the original balance in the equation and once again it's as simple as X = Y. Free will eventually upsets it again, and, repeat 6 times over up until the movies.
There were multiple versions of Neo, each of which had a version of Smith. All of which, up until the most recent Neo, were removed from the equation by The Architect (as was the Neos' choice) which removed all the Smiths from the equation too. Keep in mind that in the specific Matrix that we saw, Neo chose not to be eliminated in this fashion (and was the first to do so).
In essence: Neo and Smith are opposing forces, somewhat the same thing, somewhat both The One (although Smith could be called The Negative One).
This is the ghost in the machine of the Matrix: it was programmed to seek a balanced equation. Every time The Architect attempts to balance the equation he must add an opposite, creating these dichotomies. The Architect even created his own counter-balancing variable: The Oracle - throughout the movies she continuously upset the balance of The Matrix.