Can I modify my PS2 by adding computer parts like a GPU, a HDD or other computer peripherals?
You can modify the PS2, but only as much that Sony allows you to. You can’t change/add another GPU because the one in the PS2 is soldered on and is not easily removable. You CAN add a hard drive however, by purchasing the PS2 HDD unit and slipping it into the expansion bay. As for other computer peripherals, you can buy some USB mouse, keyboard, or microphone headsets, but they pretty much have to say that they are compatible with the PS2 before they work.
