Let's ignore the art house category (it's as good/bad as art house stuff elsewhere in the world). Let's also ignore genuine C-grade movies made for provincial markets (this category is sort of equivalent to the low-budget soft porn you see on premium cable TV channels late at night in the US).

Let's also ignore movies that are a money-laundering front for mafia bosses in Dubai who also want an excuse to hobnob with young starlets.

That leaves the mainstream A-circuit movies that are not obviously hamstrung by downmarket targeting or by by being non-serious mafia-funded intentions.

i.e.,we are talking about the sorts of movies Shah Rukh Khan and Abhishek Bachhan star in. They number maybe 100-150 a year, not 1000 (the bulk of the production is the C-grade low-budget provincial stuff). Basically the kinds of movies that the Times of India might bother to review.

This breaks down further into several categories, some of which suck, some of which don't:

1. Gritty and Hard-Boiled (Not Sucky): The gritty stuff about crime, venal politics, slumlords etc. -- this stuff does not suck. It tends to be very good, and I enjoy it. At the extreme-gritty end, you have movies like Drohkaal that are near art-house quality. In the more mainstream category, you have movies like Vaastav, Astitva, Sarkar, Ganga-Jal, Company, Guru... good stuff. Basically realistic drama done right, with little sentimentality, no redemption crap, no pandering and good acting. Must be my gloomy outlook, but this is my favorite category.
2. Satire (Not Sucky): Then there is the uniquely Indian genre of satirical comedies. I casually know Abbas Tyrewala, the screenwriter of two of the best ones in recent memory: Munnabhai MBBS and Lage Raho Munnabhai. Abbas once made a remark that Hindi (a language manufactured in 1947, by cleansing the vernacular Hindustani of Urdu influences, and bureaucratically mashing up several North Indian dialects) is really only good for one thing: vyang, a Hindi word that sort of means satire, but has connotations of sardonic wit and fatalism as well. So this is the vyang tradition. It is oftentimes livened up by a dash of slapstick. Abbas' movies, as well as older movies by people like Amol Palekar, fit this category. I'd trace this way back to Palekar's Golmaal through movies like Jaane Bhi Do Yaaron, Rangeela etc.
3. Melodrama (Really Sucky): Feel-good redemptive, escapist, melodramatic bullcrap. Unfortunately, this is also the true mainstream core and has always been. This is where Shah Rukh Khan, Hrithik Roshan and other assorted male and female mind-candy reign. Salman Khan is a regular here too, though he has played some decent comedy roles in category 2 as well. The plots are crud. The acting is beyond awful (in fact, many of the pure eye-candy type cast members like Hrithik Roshan and John Abraham basically pose and strut like models rather than even attempting to act. Shah Rukh at least tries to act, even though he ends up hamming all the way through most of the time). The themes are chosen to pander to the worst sorts of hypocrisy and double standards in Indian culture (eg. blowhard family-values rhetoric coupled with salacious wink-and-nudge sexuality... you have tropes like a heroine who has been skimpily dressed through the first half of the movie suddenly turning all traditional, wearing a sari and bindi, and touching the feet of some awful patriarch character played by Amitabh Bachhan. Why he continues to take on such roles is beyond me. He still has roles in him that don't make want to throw up (the cop in Bunty aur Babli for example), but he keeps taking on these bullshit roles. Maybe he needs the money.)
4. Patriotic (Sucky): Patriotic, jingoiistic, nationalist bullshit. Back in the day (1950s, 60s) when Raj Kapoor and Manoj "Mr. India" Kumar were doing their thing, it was schmaltzy, but tolerable. Movies like Border and Gadar started testing my patience. The new breed (Swades, Pardes, Lagaan, Rang De Basanti) I detest. Lagaan was somewhat redeemed by its entertaining cricket-match/Asterix-and-Obelix type plot, but the overall tone was awful. Aamir Khan is the prime offender here. I don't know where he went wrong. These movies tend to well produced and written, and often have fantastic soundtracks (Lagaan especially). But deep down, there is something very wrong with the philosophy and nationalist anxieties at work. The movies have a somewhat global perspective by necessity, but there is a lack of generosity of spirit in the treatment of the rest of the world. The movies seem driven by a sort of desperate need to assert cultural superiority. One exception perhaps is (surprisingly) the Shah Rukh Khan starrer, Chak de India. It displays a gentler, not-so-assertive sort of patriotism, and also addresses the very tricky issues of diversity within India and the difficulties they present in forging a national identity. As an aside, Shah Rukh is at his best when is taken out of the money-making Category 3 that he dominates like a colossus. My favorite SRK movies are the ones where he's not being SRK.
5. Middle-Class (Jury Out): There is one category where the jury is still out: this is the newish category targeting the small but growing urban, educated Indians. It has some of the self-absorption characteristic of similar movies in the US. I don't know quite what to make of it. Some of the older movies in the same vein used to be made by people like Hrishikesh Mukherjee and Gulzar, but I am talking about the new stuff targeting the post-IT boom-economy India. The biggest hit in recent times has been from this category: Three Idiots. It is a screen adaptation of 5 point someone, by Chetan Bhagat and is actually about life in my alma mater, IIT. It gets some things right, but is somewhat superficial. It also has an internal critique of the education system, and about how it is killing individuality by trying to turn everyone into an engineer. Other movies in this genre (can't recall names off the top of my head) address themes like premarital relationships, working abroad, single motherhood etc. I don't know quite what to make of this category. The quality is very patchy, and the introspection offered up ranges from deeply deluded to weirdly insightful at times. A big challenge for this category is English. Since so much of both the audience and the subject matter involves people who speak/think in English/Hinglish like me, and only switch to pure Indian-language thinking/speaking on rare occasions, these movies need to be at least partly in English to work. Few have risen to this challenge well. English in Indian movies still seems awkward. One movie that did this reasonably well is Being Cyrus.

Overall, despite the dominance of crud, there is much to be hopeful for. Production quality is rising. Acting is getting more realistic and understated instead of histrionic/theatrical. The dominant category 3 stuff is still the majority of both the movie count and revenues, but movies like 3 Idiots demonstrate that there is a growing market for stuff that tries to be different. The music seems currently trapped in a crappy phase, but I am hoping that will change once new talent of A. R. Rehman caliber emerges. There is also a new kind of acting talent rising that is very comfortable in its own skin. Actors like Abhishek Bachhan, Boman Irani, Arshad Warsi are beginning to displace the pure eye candy.

But overall, why does the sucky stuff suck at all? (I'd say it accounts for about 80% of the A-level mainstream).

Here's the answer:

There is a fascinating graph in Pankaj Ghemawat's "World 3.0" (Chapter 11, "Global Homgenization", page 239) that plots an interesting data set: self-perceived cultural superiority vs. self-perceived need for cultural protection.

The fascinating thing is that these two variables are very strongly correlated (the coefficient is 0.68). At the bottom left (low self-perceived cultural superiority and low self-perceived need for cultural protection), you have Sweden.

At the other extreme, (high perceived cultural superiority, high perceived need for protection) -- you guessed it, India. India tops the chart by a wide margin.

I agree with Ghemawat's take: "Swedes appear pretty laid-back about foreigners... whereas Indians, to my dismay, are not...what we seem to be seeing here is cultural insecurity, not aggressiveness."

On average, modern Indians -- the subset who articulate national identity -- pretend to feel superior to the rest of the world, but are deeply insecure about their self-proclaimed superiority.

That's at the heart of why the sucky stuff is sucky. Cultural insecurity. Paradoxically, when India was stuck in the doldrums of socialist governance, this effect wasn't so bad. The movies didn't suck. But since India has hit the 8% growth curve, this stuff has gotten increasingly sucky.

Why? I suspect it is because the new-found wealth and growth and IT-industry clout provide ammunition for patriotism and jingoism, but deep down, the cultural identity is still not secure, there is a general fear that the country is actually faking success and will be "found out" somehow. There is also the strong awareness of the great cognitive dissonance -- nuclear weapons, IT and a space program on the one hand, and poor people still having no toilets or sanitation on the other, living in slums and taking dumps on railway tracks for the middle class to see on their commute early in the morning.

The sucky part of Bollywood is exactly the part that goes into deep denial about this precarious adolescent development stage (truly, India is an "emerging" economy in a psychological sense, just joining the global workforce, while America is a "retiring" economy, leaving it).

So two things need to happen for Bollywood to stop sucking. First, the economy needs to get on a solid foundation where people no longer feel insecure about whether they deserve the new prosperity. This might include Indian companies doing more than menial Y2K type work or outsourced gruntwork for example.

Second, the bottom of civil society needs to be raised sharply. To oversimplify, Bollywood will continue to suck so long as there isn't adequate public sanitation in the country.

If both are achieved, India will have moved on from adolescence to adulthood. If not, it will fall back into the pre-1990 state of general crappiness all around.