Daniel Gross (Slate) explains why we shouldn't worry about the size of the Fed's balance sheet.
Most of the assets that will remain on the balance sheet the longest are the mortgage backed securities (think Fannie and Freddie). Here, the Fed can afford to be a long term speculator. They, more than your average Wall Street entity, have the patience to wait until those mortgages are refinanced, the houses are sold, or they are simply paid off. Remember, a crucial problem during the crisis was the inability to properly price the assets because no one was willing to buy. This drove down the prices of good assets as well as bad, and you know the result.
By and large, the effort by the Fed was successful. Attention should now be focused on how to make sure they don't ever need to do it again. That's a tall order when we all know that if they had to do it again, they probably would. We call this problem "moral hazard," and it's the reason that common sense regulation really is necessary.
The bottom line? The Fed wants to get the junk off its balance sheet and return to a situation in which it has about $1 trillion in assets, the lion's share of them in the form of government bonds. To do so, it will need to rid itself of about $1.3 trillion in assets. That's a lot. But when you add up the components of the balance sheet that are shrinking, the task doesn't seem quite as daunting. By the end of 2011, by my rough calculations, at least $300 billion of the Fed's current assets will be gone with a substantial additional amount on the way out--and all without the Fed having to stage a huge sale of assets.
Most of the assets that will remain on the balance sheet the longest are the mortgage backed securities (think Fannie and Freddie). Here, the Fed can afford to be a long term speculator. They, more than your average Wall Street entity, have the patience to wait until those mortgages are refinanced, the houses are sold, or they are simply paid off. Remember, a crucial problem during the crisis was the inability to properly price the assets because no one was willing to buy. This drove down the prices of good assets as well as bad, and you know the result.
By and large, the effort by the Fed was successful. Attention should now be focused on how to make sure they don't ever need to do it again. That's a tall order when we all know that if they had to do it again, they probably would. We call this problem "moral hazard," and it's the reason that common sense regulation really is necessary.

The treasury should fund Fannie and Freddie directly so that mortgage rates are at desired levels so that the GSE's won't have any hedging needs. Foreclosures and pushed down the value of all homes in the neighborhood. If you're being foreclosed on, the government should take on the mortgage and charge rent to the old owners.
Government's need to do this will highly depend on how safe the loans banks made are. That's why we need to change the rules for banks so that they go back to keeping risky borrowers away with a 10 foot pole. Banks should no longer be allowed proprietary trading, secondary market transactions, buy credit default swaps. There is no reason banks should make loans that it doesn't feel comfortable having on its balance sheet. Banks exist for one reason, to make loans that will be repaid.
It should be recognized that taxes regulate aggregate demand. The crisis could have been contained more quickly if taxes were decreased sharply and quickly, including a payroll tax holiday. Federal aid to states should have been more substantial.
TARP was a disaster. It didn't fix the problem of getting lending restarted. It would have been much more effective if it was given to lenders so they could repay loans. That would have helped the bank the same, and help promote aggregate demand which had fallen because people/businesses want to repay loans. I'd rather loose the $700B and get something for it rather then live trillions below potential GDP.
A payroll tax holiday is something that a lot of economists would have supported, I think. I seem to recall some prominent folks mentioning it at the time, though I can't recall who right now. I would have had mixed feelings. I think it would have had an immediate positive impact, yes... but would have also set a troubling precedent. I also think it would weaken the impression people have of the government's commitment to Social Security. If the payroll tax is politically negotiable, then people will want to negotiate.
TARP was certainly not perfect, but it is unclear that your alternative would have been better. (I'm assuming you meant "given to borrowers so they could repay loans" since TARP gave money to the banks.) Giving money to the borrowers would be much more costly to administer and subject to potential for fraud and abuse.
"Federal aid to states should have been more substantial." No argument there. And it's not too late to do more. Yet.