Your local camera shop will most likely have paper backdrops for sale. Mine sold me 9ft of 10ft wide paper for $35.
I hang it from rail of PVC pipe that cost me about $5 in parts to build.
I started with $25 work lights from home depot. It's harsh lighting but I diffused it by hanging super thin white cloth in front to soften it up quite a bit. I used two lights, one(key) light to provide the main light, then another light with slightly thicker cloth to make it less bright as the fill light, then I mounted a halogen bulb above on the rail for hair lighting.
Everything together for my basic continuous lighting setup was less than $100.
I even used a walmart tripod and invested in an opetka wireless remote. +$45 for those two.
Now I've moved on to strobing(using flashes mounted on stands), but it worked well for a start.
Besides, learning how to setup your lighting is half the battle.