Here I am not talking about building everything on your own. As I mentioned previously, whenever possible and as much as it makes sense, you should probably rely on 3rd party tools and services.
But ... there are things you absolutely must not be giving up control of.
Make sure you can't be locked out of anything.
The highest risk is with individuals, being contractors, employees, ... A simple question should help you determine if you're safe or not. Will I loose access to anything if I have a fallout with X ?
If there is a yes, then, you should prioritize doing something about it as soon as possible.
Make sure you own your assets, it can be code, websites, domain names, ... Anything that you use and build, you must be the owner. Especially anything related to your brand.
Looking at job posting on sites like upwork, you can regularly find job ads from people not being able to access their server, who loose their domain name, ... because they can't reach the person that was there initially to do the work. They entrusted a 3rd party with some critical aspects. That person disappeared and now, there is no way they can safely retrieve control. Don't make that mistake.
Buy what you need and give access to the people who need it, or have them purchase and transfer asap. Just, for your own sake, don't be content with having someone else have ownership.
Same thing with 3rd party tools and services, you may not be able to own them, but make sure that you can't be locked out of them by anyone (besides platform risk).
The easiest way to do so is to make sure you're the owner/administrator of the account. Then grant access to whoever needs it, with individualized permissions. Those permissions should be as restrictive as possible.
Finally, when cutting the relationship with a contractor, employee, provider of any kind, disable their access. Especially if there may be bad blood between the both of you. Don't open yourself to any possible retaliation.
Sometimes, it is not possible, or desirable, to have the control. It can take time and resources to follow up on every possible item, and you may have no time for it.
When that happens, make ensure at least to receive a backup of everything, on another location you control, so that whatever happens, you can get access to the data / source code / ... it may be troublesome to recover later on, but you wouldn't loose everything and start from scratch.
Whatever you do, own your domain name...
Once you own or control everything, you need to make sure that you build a sound handoff strategy. Meaning, making sure that someone else could take over everything if needed.
Your business should be the owner, not you the person.
It can necessary for both positive or negative outcome. If you sell your business, go on vacation, or something happens to you, ... someone else will have to take over. You should aim to make the transition process as easy and seamless as possible. They will have a lot of things to think about, the transition will be much better if they don't need to fight to access critical components.