According to your diagram, you cannot mess up the wiring. Reverse the two wires at the switch and they still complete the circuit. Reverse the two wires at the bell and they still complete the circuit.
Connections to this type of transformer are basic-three wires into the high side (H1, H2, and H3) and three wires out from the low side (X1, X2, and X3). No. 10 AWG primary-side wiring (22.6A at 480V) and secondary-side No. 8 (45.1A at 240V) will give you full use of the transformer capacity.
Connect the wires. Once the new transformer is mounted, attach your household power wires to the matching wires on the new transformer using wire nuts. Then connect your existing doorbell wires to the two screw terminals. It doesnt matter which wire goes to which terminal.
Use a voltage tester to make sure the power is off. One side of the transformer has the 120 volt wiring attached to it. The low voltage side will have two small wires attached to it. The polarity for these two wires does not matter.
For Transformer, we should choose two types of cable such as the primary side and secondary side. The rating of the cable should be 150% of the full load current. i.e if your transformers primary current is 450 Amps means, you should choose 450 *150% = 675 Amps capacity size cable.