That sounds to me like internal parasites. Some parasites are visible from the outside (look for any odd white or other strange-colored spots on him that are not a normal part of his coloration). Internal parasites are not visible because they live in the digestive tract of the fish, only showing through the anus (if that's what it's called in a fish) when the female parasitic worm hangs her end out to lay eggs into the water. Some common symptoms of parasites are lethargy, unexpected weight loss, wild and erratic swimming around the tank, and the fish "flashing" - swimming swiftly against objects in the tank to help scratch itself and ease the discomfort. From experience with a past female that had parasites, it sounds like your guy's doing the erratic swimming and may be flashing as well. Fish-affecting parasites are always present in the water - even tap water. They can't effect us because they need the constant water to survive, but they can affect fish whose immune systems have been compromised.
There are several things to get rid of this:
First, make certain the fish's temperature is between 75-80 degrees. While warmer temps will not kill the internal parasites, it will speed up their life cycle a bit and will aid your fish's metabolism and immune system to get rid of them. Lamps are not an appropriate solution to this: their heating is sporadic, the light causes stress, and temperatures drop the moment the source goes away. If you need to warm up your Betta's water, and your Betta's tank is not large enough for an aquarium heater, you should get him into an appropriate one, preferably at least 2 gallons, the minimum amount of water for even the smallest heaters.
Second, which is VERY important, is to get the fish into medication! Some external parasites can be killed just with freshwater salt (see the next paragraph), but internals are much, much more of a problem and basically always require medication. I've had success with Jungle's Parasite Clear, a common brand available in most pet shops that sell fish. Follow the instructions carefully, and don't touch the medicine yourself. If the tank is cycled, you can follow the dosage exactly as stated. My tanks aren't, so I changed the water 100% every two days and put medication in the new water to bring it back to the level of the old water. Keep in mind that all instructions on fish meds are made for cycled tanks - even with filter pads removed, some of the medication gets neutralized in the filters. The levels build much faster and do not dissipate in unfiltered tanks. my female with parasites took 4 rounds of Parasite Clear (2 48-hour treatments at a time, with space between them) over the course of about a month before her symptoms no longer returned. Internal parasites are more stubborn and are difficult to get rid of, but they are worth it. Fish are much healthier and more comfortable without them!
Third...use freshwater aquarium salt! FRESHWATER! Salt intended for marine (saltwater) aquariums is not the same and will damage your fish. Freshwater salt adds electrolytes, kills bacteria, aids in gill function, and is despised by fish parasites.
Fourth - keep the water crystal clear! It might be best to remove any substrate (gravel, marbles, et cetera) from the tank during this (ONLY in uncycled tanks!), unless your fish uses them as a resting spot and you feel it'd cause him more stress. Parasites lay their eggs into the water, which sit on the floor of the tank until they hatch and find a host. Losing the gravel will give them less places to sit and will make cleaning the bottom during 100% water changes much easier. Internal fish parasites can't survive more than a couple days without water, so letting the gravel dry for several days should kill them. I personally would run it under hot water first as well.
Fifth - antibiotic! Fish are very prone to bacterial infections when recovering from parasites, particularly in parts where the worm had been latched onto them after the pest comes loose. Always use antibiotics with antiparasite medication. Jungle has an Fungus Clear med that is also an antibiotic and is safe to use with Parasite Clear. I've had success using Tetracycline with Parasite Clear as well. If doing the 2-day changing routine like I did, it's best to put the antibiotic in at the same time as the antiparasite or two days alter, so the cleaning schedule is still the same for both.
Fish parasites are stubborn, gross, and very unfriendly and dangerous for the fish. You should see a definite improvement in your boy's behavior after the meds have killed the worms and any that hatch from eggs they may have laid.
If you have any other questions, you can also check for help on www.ultimatebettas.com, where I've gotten lots of help with my fish. Good luck!