I used to suffer from severe insomnia 12-13 years ago. Medication didn't help. What did the trick for me was voluntarily quitting the pills, adopting much better "sleep hygiene" and slowly making lifestyle changes.

It is the only real long-term solution I think. There are physical hygiene behaviors and mental hygiene behaviors, as well as cognitive reframing steps you need to take. These all take time to work (like 4-6 months minimum) but do work.

Physical:

1. Don't read or watch TV in bed. Definitely never eat or drink in bed. Beds are only for sleep and sex.
2. Don't go to your bed (or even bedroom) until you are actually sleepy. Even if that translates to 5:30 AM. If you live in a studio, put up a wooden screen.
3. No matter how late you go to sleep, try to wake up at roughly the same time. Use an alarm to train your brain initially if you need to. Yes, this means even if you go to sleep at 5:30, you still wake up at 7 or 8 or whatever time you pick. When you do get up, make it a habit to open the blinds or step out and get at least 20 minutes of sunlight. This is biologically key: our main 24-hour body clocks are entrained to sunlight, and morning light resets and tunes the clock. Artificial light won't do until it is specifically designed for the purpose, because the wavelengths that affect your chronobiological sensors in your eyes aren't the same ones that you use for vision. Sunlight has these wavelengths, most artificial light sources don't.
4. Don't eat too close to bedtime. Ideally stop 3 hours before.
5. Try and get some light exercise a couple of hours before you go to bed, like taking a walk. Regular heavier exercise earlier in the day is much better, but that can be hard to keep up.
6. Develop a sleep ritual: brushing teeth, pajamas, making the bed a bit if it is too messed up. Make the ritual more pleasantly elaborate and slow if necessary. Try to do this ritual at the same time, even if you actually go to bed at different times.
7. If you go to bed but still toss and turn for an hour or more, get out. If the problem isn't mental, certain snacks will help: cereal with milk or plain bread for instance.
8. Turn down the heat. A few degrees of chill can do wonders.
9. If all else fails, try moving to the couch. Last resort measure.
10. If you like gadgetry, there's some good stuff out there. I am personally considering buying this one: http://www.myzeo.com/

Mental:

If you can't let go runaway thoughts, try these things to shut them off:

1. Write them down on a pad near your bed if they are only minor close-ended niggles about to-do stuff. Learn GTD if you want to take it to the next level.
2. Get out of bed and go to your desk and think them through on paper if they are complicated and you are alert enough to actually process them usefully. A pre-emptive way of doing this is to notice mind-grabbing thought trains coming on earlier in the evening, and going for a walk to help process them and calm them. No later than around 8-9 PM though. Really late walks, even if safe, can reinforce your awareness of your insomnia and its socially exceptional nature
3. If they won't let go and you aren't alert enough to process, simply sit down somewhere else in the dark, like a comfortable armchair, and let the thoughts run wild until they run out of steam and exhaust themselves. I call this "giving my thoughts enough rope to hang themselves." Don't attempt to stop them. This move goes well with a midnight snack.
4. Runaway thoughts have an inside-head locus. A good way to shut them down if all else fails is to get online and catch up on the news for about an hour or so. It shifts your cognitive center of gravity to an external bitstream from an internal bitstream.
5. Be very wary of TV at night. It's a wildcard in mental sleep hygiene. It can help sometimes, make things worse sometimes, and tends to be very situation dependent.
6. Develop a hobby that requires delicate work with your hands, like building model ships. Many serious insomniacs are way too cerebral. For me, it is photorealistic drawing. I am planning to get into model ships too.
7. I'll throw in social hygiene here as well: if you are a poor sleeper, either live alone or with apartment-mates whose lifestyles are compatible. Good sleepers can ruin bad sleeper's lives. I lived with a guy who had awful sleep hygiene, who worked random schedules and was very loud when he was at home. It didn't matter to him because he could fall asleep in 5 minutes anytime, but it did a number on my routine. He's still a good friend, but I got out of that living situation as quickly as possible. If you are married or have a live-in significant other (or planning to get into that situation), take care to make adequate arrangements to protect your sleeping routine.

All this only works if your runaway thoughts are all over the place and not particularly depressive. During my worst insomnia, anything could keep me up, from wondering about life on other planets to an idea for a startup, a tricky math problem, or a speculative theory about something.

If your runaway thoughts are always about your personal life, relationships, how your life sucks, turn to suicide on occasion, and seem to go in circles rather than in wildly branching explorations that get somewhere, you may have chronic clinical depression rather than simpler personality-driven/thinking style conditions.

Seek professional help. But be a very alert patient. The medical profession sucks at dealing with this, so you have to compensate for their incompetence and apathy. They want to treat McInsomniacs or McDepressives as cheaply as possible, and this often means temporary band-aids. You want to improve your actual well-being.

Make sure your doctor isn't rushing to valium or prozac. Track the meds prescribed and check to see if they actually work. Some of these drugs can make insomnia worse in a minority of patients. Even if it IS genuine depression, try to get away from pills as soon as possible. Don't be afraid to quit the medical approach if it isn't working for you. The profession's hit rate is lousy.

Cognitive Reframing

Unless your lack of sleep is severely interfering with your waking life, understand that it is okay to have a different sleep profile. It is in fact often a sign of a strength -- the ability to think deeply.

Too many people rush to diagnose things like depression, including doctors. In my (non-professional) opinion, there's a far wider range of normal variation due to simple personality type effects. I speculate that Myers-Briggs intuitives are much more prone to this than sensory types for example.

Even if this is untrue, I do believe there are simply certain types of personalities who are not very prone to long, deep thoughts, who can fall asleep within 5 minutes. For them, insomnia is an exceptional condition when they are undergoing mental trauma around relationships or something. But for the other type, this is normal, everyday stuff.

If you like deep thinking, your brain-flywheel simply has a lot more inertia than that of people who don't. You should expect it to take longer to slow down to a halt for exactly the same reason that light motorcycles can stop in 10 feet, but big semi-trucks can take 100 yards. For me, I consider 30-90 minutes normal. Makes me an outlier but I am fine with it.

At a broader level, recognize that our environment is seriously messed up. Our bodies are designed for sunrise-to-sunset lighting, and around behaviors like napping. Today we have 24 hour electric lighting and work schedules that prevent napping. This affects some people a lot more drastically than others.

So if you want to life-hack your schedule to allow you to nap etc., go right ahead.

A recommended book: Russel Foster, Rhythms of Life. A somewhat demanding pop-science book on chronobiology. Well worth the read. It will give you a much better appreciation of the complex clocks your body runs on, 100x more complex than the most intricate swiss watch.

http://www.amazon.com/Rhythms-Li...