How much linux is required for the phone to run?

[Subsentient]

I'm sure I'm not the only one who wishes they can write their own OS

I've written code for an Arduino to communicate with a qualcomm chip/modem to receive and send SMS using AT commands. 
I wanted to try a from scratch for the raspberry pi but it seems like I'll get poor video performance because there's no documentation for using their GPU/video

I was wondering if there's a lot of linux that's required to get the phone running and if not would it be possible to write our own OS or embeded like as people may do for the Arduino?

You should definitely keep using the Linux kernel, but that is the only component you are strictly going to need. You could write an entire custom user space for the PinePhone if you really wanted to, although it will take you until you're old and gray. If you tried to rewrite everything that's needed to get phone calls working, the task really is just so monumental that I strongly suggest you don't bother trying at all.