How can I do a conditional regex replace?

Question

Language is javascript.

Example 1 - input:

I have one (1) cat

desired output:

I have one (1) cat

Example 2 - input:

I have one1 cat

desired output:

I have one cat

I want to remove any numbers in the string that are by themselves (like the second example) but not numbers that are wrapped in parenthesis (like the first example). How can I accomplish this?

Answer 1

Improving the deleted answer from blex I got

const str = '123)I have one (1) cat. I have one1 cat4). I (6have (999) cats. I hav3e a cat (123';
//   
console.log( str.replace(//(([^(\d]|^)(\d+)|(\d+)([^)\d]|$))/gm, '$2$5') )

Output

)I have one (1) cat. I have one cat). I (have (999) cats. I have a cat (

Explanation(since the nice Regex101-Explanation was deleted, too):

find digits with a leading char, that is not a digit or '(', or digits with a trailing char, that is not a digit or ')'. Replace matches with the leading/trailing char

EDIT

added start/end of line test