Evaluating trades becomes a topic of interest as each trade is made. Before the trade we're analyzing the rumors of what teams would be giving up and getting in return. Some people are content to make baseless claims about the quality of the trade when it happens, during the contracts and after it happens. Some are content to look at a few numbers like batting average, RBI and pitcher wins and leave reach a conclusion. Others look at advanced metrics, WAR in particular, to evaluate a trade when it happes, during the contracts and after. Projections are central to evaluating a trade when it happens.
I used to believe that using projections at the time of the deal was not only the best way to evaluate a trade, but the only responsible way to do so. I argued we had certain information, or rather the GMs had certain information, and that it was at the time the trade happened; not after. I was basically saying that we can't fault a GM for a bad trade if it seemed like a decent deal at the time despite him trading away a star. I believed this for quite awhile.
I've since realized the assumption that it's the only way to evaluate a trade is based entirely on the parties involved having the same information as the public had. But they have a lot more information at their disposal and all the teams, an organization that leans heavily on sabermetrics or not, use whatever information they have when considering a trade.
Every year fans want their favorite teams to fire their GM and hire some hot shot that might available. The reason for this is relatively simple: not all GMs are created equal. They can't possibly be viewing the players involved in a trade in the very same manner. They see things differently. They might project one player better than another team does. It's because of this that evaluating a trade at the time it's made is only one way to evaluate a trade.
Suppose two teams swap equally paid players who made an equal amount of money. Their projections at the time of the trade are identical. These trades are rare, but this is just an example. At the time of the trade, what went one way was equal to went the other way. During that season one player was worth 2 WAR while another was worth 3 WAR. It's no longer equal. It's fairly obvious one team got the better end of this trade. Was it a bad trade?
Most of the time it's not as simple as this to evaluate. You usually have players making different amounts of money being traded for one another. What if Team A gets 9 WAR in a trade over, say, 3 years while Team B also gets 9 WAR. We might immediately think this trade was a wash, but players are paid real money and that money paid to them is money not paid to other players. Team A has paid this guy $40 million while Team B paid only $12 million. Obviously Team B is the winner of this trade, but does the winner and loser really matter? Is it that important to us that we have to think of trades in terms of winners and losers?
What is it that we are ultimately trying to find? I can't speak for everyone, but I'm most interested in the rationale for the trade and after the contracts are up I'm curious how each team did in terms of projecting the players they acquired. Who won the trade is irrelevant to me because the simple fact is that even a good GM is going to lose trades from time to time. A bad GM will win them too.
If we have a large enough sample of trades we could evaluate how well one GM is at projecting his own talent vs. talent he acquires from another organization. How large does the sample have to be? I don't have the foggiest idea. I know it's not 1 trade or even 3 or 5 trades. It's not 1 years, 2 or 3 years. It's probably not even 4 or 5 years either, but I don't really know. If I was going to research this I might set the cutoff at 6 years. At that point maybe we can spot which GMs are better than others at trading. However, it's not only the GM that's involved in trade discussions.
Getting back to when to evaluate trades, there are two different times at which this can be done. Both should be done because each method gives us information that is valuable to us. We'd probably want to put more weight on the outcomes of the trades, but we can't reach too many conclusions about any individual trade other than this: one team got the better end of the deal. One trade isn't necessarily reflective of the skills of the people involved in the trade discussions. Over many trades it may very well be, but most GMs don't stick around long enough to have that kind of track record.