Model architecture similar to GPT-2 small but parameter count mismatch

[Dimagidhp]

Hi, I’m experimenting with the architecture using the same configuration as GPT-2 small , but I'm noticing that the total number of parameters does not match the expected count for a GPT-2 model. I'm using the following configuration:
Embedding dimension: 768
Hidden dimension: 3072
Number of layers: 12
Number of heads: 12
Is there something specific in the architecture (e.g., embedding layers, heads, layer norm, etc.) that might explain the difference? Or is the model structure not meant to exactly mirror GPT-2?

Any insight would be greatly appreciated!

Thanks in advance.