Since the power went out all at once without any warning, and then turned back on again after kicking the battery pack, that is indicative of an intermittently loose connection, broken wire, faulty switch or circuit breaker, or other similar problem. Most speed controllers are 100% solid state devices that do not have any mechanical parts inside that would turn on or make contact if the controller was kicked or vibrated so the problem is most likely not going to be the speed controller.
It does not sound like the battery pack is the problem because you were able to ride the scooter for two miles after it turned on again. It could be the key switch or another electromechanical part like the circuit breaker that is prone to making and breaking contact from vibrations.
Because only the power to the motor went out the first time, and the power to everything went out the second time, it is sounding more like a key ignition switch problem to me because the key switch is the one part that controls all of those different functions.
Unfortunately the parts lists we have for the TaoTao ATE 501 and Xtreme XB 600 electric scooters do not show which key switches they use. There are often subtle differences between key switches that look identical so I would make sure that the wire length and connector is the same and that the new key switches lock pin sticks out of the key switch body the same amount as it does on the old key switch.
I had to go onto my laptop to see the "insert image" icon. I'm sending the photos of the Xtreme XB600 ignition switch. The ignition switch I just bought for my old scooter (AVH1 or TAO TAO) is a 3 position, 2 wire, metal switch but the one from old scooter has the white connector while the one I'm trying to replace has a 3 position, 2 wire, it does not have a white connector that I can see. The wiring runs down the front of the bike into the floor board of the scooter where the battery pack is located. If you need to see photos of the one I bought for my old scooter that I want to use to replace the possible bad one I can take more photos.
You'll notice in one of the photos there is a red wire and a white wire with electrical tape then the wires go to red and black...not sure why that is. I thought maybe it had been spliced at one time.
Our key switch item # SWT-238 looks very close to the Xtreme XB-600 key switch in the photos that you attached. Especially where it mounts to the scooter and the length of its lock pin. Its has an 18mm center to center distance between its two mounting holes and its lock pin protrudes 12mm in the unlocked position and 18mm in the locked position.
I just checked stock and someone purchased our last SWT-238 key switch a couple of days ago however we will have more of them back in stock in around a week.
I see in your photos that the old key switch was cut and spliced into the wiring harness so it has already been replaced before. The new key switch can be cut and spliced back into the harness again to install it.
If you would like I could send you an email when the SWT-238 is back in stock?
Please let me know if you have any questions.
you may also need to inspect all connectors leading to the controller. pull them apart one at a time and look for any signs of corrosion and clean them up and put the back together. you might also want to check and see if there is a circuit breaker/power switch under the seat and make sure that the controller power wire didn't fall out.