Despite recent reports suggesting Warren Buffett was open to filling a potential void in the housing finance market if Fannie Mae and Freddie Mac were wound down, Buffett and Charlie Munger all but squashed that notion during the Berkshire Hathaway annual shareholder meeting on Saturday.

During the famous day-long Q&A session with Berkshire shareholders, a questioner asked if Buffett thought Berkshire could play a large role in a more privatized housing finance market because of Berkshire's underwriting expertise and experience in the housing market. 

Buffett said it was very unlikely that Berkshire would become a big player in that market.

So what does Buffett think should happen to the market?

Buffett highlighted the U.S. housing market's enormous size ($11 trillion) as a massive challenge to privatizing the market simply because there isn't enough capacity in the private insurance market.

He seemed to believe a system similar to the plan proposed by Sens. Johnson and Crapo, as well as Sens. Corker and Warner, could potentially work. Noting that the 30-year fixed rate mortgage is a great thing for the average American, Buffett said the government ultimately needs to be the insurance entity to keep rates low. 

His partner, Charlie Munger, took a more blunt stance.

Munger believes the current system is best option: Continue to conservatively run Fannie Mae and Freddie Mac and do not return the entities to the hands of private shareholders. Despite addressing the common concerns with the current model, Munger expressed even more doubt that private mortgage finance players would operate a better system. 

One of the largest drivers of the collapse of Fannie Mae and Freddie Mac was the pressure to deliver continued earning-per-share growth for the private shareholders. Both companies aimed to achieve this growth by taking out cheap debt and building massive securities portfolios.

These legacy portfolios are currently being wound down -- a move Buffett and Munger agreed was the right one. They said the two GSEs were essentially operating as the country's largest hedge funds before the housing crisis.

As current common and preferred shareholders continue to hang in a state of financial-limbo, the path to significant profits (outside of potential legal settlements) continues to look like one littered with road-blocks. Any passable law making its way around Congress ignores these shareholders and now two of the best business minds in the world in Buffett and Munger agree that returning these entities to private hands would be a mistake.

Munger wrapped up the discussion by saying that the old Fannie Mae and Freddie Mac "experiment was a total failure."

Read about the entire Berkshire Hathaway Q&A session here!