Jamieson, Fausset and Brown on Exodus 34:4

4. Moses ... took in his hand the two tables of stone--As Moses had no attendant to divide the labor of carrying them, it is evident that they must have been light, and of no great dimensions--probably flat slabs of shale or slate, such as abound in the mountainous region of Horeb. An additional proof of their comparatively small size appears in the circumstance of their being deposited in the ark of the most holy place (Ex 25:10).