Replacing patterns

Question

I am trying to build a regular expression that covers all these rules but there I am having trouble. Someone can help me, please? I did by brute force but it is slow. The rules are the following:

if there are some words chain in Camel case ( Camel case ) or a lower case( lower case ) on the beginning of the phrase, without being followed by a : character, and there is an empty line under and above the chain of words, then it will convert the words' chain into CAPSLOCK followed by the : character

if there are some words chain in 'CAPSLOCK' in the beginning of the phrase NOT followed by a ':' character, and in the next line is the continuation of the text, that is, that there are no other words chain in the beginning of the line followed by a : character, then concat a : character and the text right after the : character.

otherwise if there are some words chain in CAPSLOCK in the beginning of the phrase NOT followed by a ':' character and the next line there is another chain of words that are at the beginning of the line followed by an : character just remove the chain of words and replace it with a space.

also, replace all the \\n * * * \\n by a space and replace all the (Medical Transcription Sample Report) by an space too

One file exemple: SAMPLE TYPE / MEDICAL SPECIALTY:Speech - Language
SAMPLE NAME: Speech Therapy - Discharge Summary - 2

DESCRIPTION:The patient was referred for outpatient skilled speech therapy, secondary to right hemisphere disorder, status post stroke. The patient attended nine outpatient skilled speech therapy sessions.  
(Medical Transcription Sample Report)  

* * *

The patient is a 69-year-old female, who was referred for outpatient skilled
speech therapy, secondary to right hemisphere disorder, status post stroke.
The patient attended nine outpatient skilled speech therapy sessions from her
initial evaluation on 12/01/08 to her last session on 01/09/09.  

The patient made some progress during therapy. She accomplished two and a half
out of her five short-term therapy goals. We did complete an oral mechanism
examination and clinical swallow evaluation, which showed her swallowing to be
within functional limits. The patient improved on her turn taking skills
during conversation, and she was able to listen to a narrative and recall the
main idea plus five details after a three-minute delay independently. The
patient continues to have difficulty with visual scanning in cancellation
task, secondary to her significant left neglect. She also did not accomplish
her sustained attention goal, which required her to complete tasks greater
than 80% accuracy for at least 15 minutes independently. Thus she also
continued to have difficulty with reading, comprehension, secondary to the
significance of her left neglect. The patient was initially authorized for 12
outpatient speech therapy sessions, but once again she only attended 9. Her
last session occurred on 01/09/09. She has not made any additional followup
sessions with me for over three weeks, so she is discharged from my services
at this time.  

KEYWORDS:speech - language, outpatient speech therapy, swallow evaluation, swallowing, skilled speech therapy, hemisphere disorder, speech therapy, speech

Would be :

 SAMPLE TYPE / MEDICAL SPECIALTY:Speech - Language  
SAMPLE NAME: Speech Therapy - Discharge Summary - 2 

    DESCRIPTION:The patient was referred for outpatient skilled speech therapy, secondary to right hemisphere disorder, status post stroke. The patient attended nine outpatient skilled speech therapy sessions.    The patient is a 69-year-old female, who was referred for outpatient skilled speech therapy, secondary to right hemisphere disorder, status post stroke. The patient attended nine outpatient skilled speech therapy sessions from her  initial evaluation on 12/01/08 to her last session on 01/09/09. The patient made some progress during therapy. She accomplished two and a half out of her five short-term therapy goals. We did complete an oral mechanism examination and clinical swallow evaluation, which showed her swallowing to be   within functional limits. The patient improved on her turn taking skills
    during conversation, and she was able to listen to a narrative and recall the main idea plus five details after a three-minute delay independently. The patient continues to have difficulty with visual scanning in cancellation task, secondary to her significant left neglect. She also did not accomplish her sustained attention goal, which required her to complete tasks greater than 80% accuracy for at least 15 minutes independently. Thus she also continued to have difficulty with reading, comprehension, secondary to the significance of her left neglect. The patient was initially authorized for 12
    outpatient speech therapy sessions, but once again she only attended 9. Her last session occurred on 01/09/09. She has not made any additional followup sessions with me for over three weeks, so she is discharged from my services at this time.  

    KEYWORDS:speech - language, outpatient speech therapy, swallow evaluation, swallowing, skilled speech therapy, hemisphere disorder, speech therapy, speech

another example:

SAMPLE TYPE / MEDICAL SPECIALTY:Surgery  
SAMPLE NAME: Tonsillectomy & Adenoidectomy - 3 

DESCRIPTION:Tonsillectomy and adenoidectomy. Obstructive adenotonsillar hypertrophy with chronic recurrent pharyngitis.  
(Medical Transcription Sample Report)  

* * *

PREOPERATIVE DIAGNOSIS: Obstructive adenotonsillar hypertrophy with chronic recurrent pharyngitis.  

POSTOPERATIVE DIAGNOSIS:Obstructive adenotonsillar hypertrophy with chronic recurrent pharyngitis.  

SURGICAL PROCEDURE PERFORMED:Tonsillectomy and adenoidectomy.  

ANESTHESIA:General endotracheal technique.  

SURGICAL FINDINGS: A 4+/4+ cryptic and hypertrophic tonsils with 2+/3+ hypertrophic adenoid pads.  

INDICATIONS:We were requested to evaluate the patient for complaints of enlarged tonsils, which cause difficulty swallowing, recurrent pharyngitis, and sleep-induced respiratory disturbance. She was evaluated and scheduled for an elective procedure.  

DESCRIPTION OF SURGERY: The patient was brought to the operative suite and placed supine on the operating room table. General anesthetic was administered. Once appropriate anesthetic findings were achieved, the patient was intubated and prepped and draped in the usual sterile manner for a tonsillectomy. He was placed in semi-Rose ___ position and a Crowe Davis-type mouth gag was introduced into the oropharynx. Under an operating headlight, the oropharynx was clearly visualized. The right tonsil was grasped with the fossa triangularis and using electrocautery enucleation technique, was removed from its fossa. This followed placing the patient in a suspension position using a McIvor-type mouth gag and a red rubber Robinson catheter via the right naris. Once the right tonsil was removed, the left tonsil was removed in a similar manner, once again using a needle point Bovie dissection at 20 watts. With the tonsils removed, it was possible to visualize the adenoid pads. The oropharynx was irrigated and the adenoid pad evaluated with an indirect mirror technique. The adenoid pad was greater than 2+/4 and hypertrophic. It was removed with successive passes of electrocautery suction. The tonsillar fossa was then once again hemostased with suction cautery, injected with 0.5% ropivacaine with 1:100,000 adrenal solution and then closed with 2-0 Monocryl on an SH needle. The redundant soft tissue of the uvula was removed posteriorly and cauterized with electrocautery to prevent swelling of the uvula in the postoperative period. The patient's oropharynx and nasopharynx were irrigated with copious amounts of normal saline contained with small amount of iodine, and she was recovered from her general endotracheal anesthetic. She was extubated and left the operating room in good condition to the postoperative recovery room area.  

Estimated blood loss was minimal. There were no complications. Specimens
produced were right and left tonsils. The adenoid pad was ablated with
electrocautery.  

KEYWORDS:surgery, obstructive adenotonsillar hypertrophy, pharyngitis, tonsillectomy, adenoidectomy, uvula, obstructive, adenotonsillar, hypertrophy, hypertrophic, fossa, tonsils, oropharynx, electrocautery, pads

would be:

SAMPLE TYPE / MEDICAL SPECIALTY:Surgery  
SAMPLE NAME: Tonsillectomy & Adenoidectomy - 3 

DESCRIPTION:Tonsillectomy and adenoidectomy. Obstructive adenotonsillar hypertrophy with chronic recurrent pharyngitis.  

PREOPERATIVE DIAGNOSIS: Obstructive adenotonsillar hypertrophy with chronic recurrent pharyngitis.  

POSTOPERATIVE DIAGNOSIS:Obstructive adenotonsillar hypertrophy with chronic recurrent pharyngitis.  

SURGICAL PROCEDURE PERFORMED:Tonsillectomy and adenoidectomy.  

ANESTHESIA:General endotracheal technique.  

SURGICAL FINDINGS: A 4+/4+ cryptic and hypertrophic tonsils with 2+/3+ hypertrophic adenoid pads.  

INDICATIONS:We were requested to evaluate the patient for complaints of enlarged tonsils, which cause difficulty swallowing, recurrent pharyngitis, and sleep-induced respiratory disturbance. She was evaluated and scheduled for an elective procedure.  

DESCRIPTION OF SURGERY: The patient was brought to the operative suite and placed supine on the operating room table. General anesthetic was administered. Once appropriate anesthetic findings were achieved, the patient was intubated and prepped and draped in the usual sterile manner for a tonsillectomy. He was placed in semi-Rose ___ position and a Crowe Davis-type mouth gag was introduced into the oropharynx. Under an operating headlight, the oropharynx was clearly visualized. The right tonsil was grasped with the fossa triangularis and using electrocautery enucleation technique, was removed from its fossa. This followed placing the patient in a suspension position using a McIvor-type mouth gag and a red rubber Robinson catheter via the right naris. Once the right tonsil was removed, the left tonsil was removed in a similar manner, once again using a needle point Bovie dissection at 20 watts. With the tonsils removed, it was possible to visualize the adenoid pads. The oropharynx was irrigated and the adenoid pad evaluated with an indirect mirror technique. The adenoid pad was greater than 2+/4 and hypertrophic. It was removed with successive passes of electrocautery suction. The tonsillar fossa was then once again hemostased with suction cautery, injected with 0.5% ropivacaine with 1:100,000 adrenal solution and then closed with 2-0 Monocryl on an SH needle. The redundant soft tissue of the uvula was removed posteriorly and cauterized with electrocautery to prevent swelling of the uvula in the postoperative period. The patient's oropharynx and nasopharynx were irrigated with copious amounts of normal saline contained with small amount of iodine, and she was recovered from her general endotracheal anesthetic. She was extubated and left the operating room in good condition to the postoperative recovery room area. Estimated blood loss was minimal. There were no complications. Specimens produced were right and left tonsils. The adenoid pad was ablated with electrocautery.  

KEYWORDS:surgery, obstructive adenotonsillar hypertrophy, pharyngitis, tonsillectomy, adenoidectomy, uvula, obstructive, adenotonsillar, hypertrophy, hypertrophic, fossa, tonsils, oropharynx, electrocautery, pads

Third exemple:

  SAMPLE TYPE / MEDICAL SPECIALTY:Urology  
SAMPLE NAME: Laser Vaporization of Prostate 

DESCRIPTION:Cystopyelogram and laser vaporization of the prostate.  
(Medical Transcription Sample Report)  

* * *

PREOPERATIVE DIAGNOSIS:Benign prostatic hypertrophy.  

Postoperative Diagnosis:Benign prostatic hypertrophy.  

SURGERY: Cystopyelogram and laser vaporization of the prostate.  
amenesia:Spinal.  

ESTIMATED BLOOD LOSS:Minimal.  

FLUIDS:Crystalloid.  

BRIEF HISTORY:The patient is a 67-year-old male with a history of TURP, presented to us with urgency, frequency, and dribbling. The patient was started on alpha-blockers with some help, but had nocturia q.1h. The patient was given anticholinergics with minimal to no help. The patient had a cystoscopy done, which showed enlargement of the left lateral lobes of the prostate. At this point, options were discussed such as watchful waiting and laser vaporization to open up the prostate to get a better stream. Continuation of alpha-blockers and adding another anti-cholinergic at night to prevent bladder overactivity were discussed. The patient was told that his symptoms may be related to the mild-to-moderate trabeculation in the bladder, which can cause poor compliance.  

The patient understood and wanted to proceed with laser vaporization to see if
it would help improve his stream, which in turn might help improve emptying of
the bladder and might help his overactivity of the bladder. The patient was
told that he may need anticholinergics. There could be increased risk of
incontinence, stricture, erectile dysfunction, other complications and the
consent was obtained.  

PROCEDURE IN DETAIL: The patient was brought to the OR and anesthesia was applied. The patient was placed in dorsal lithotomy position. The patient was given preoperative antibiotics. The patient was prepped and draped in the usual sterile fashion. A #23-French scope was inserted inside the urethra into the bladder under direct vision. Bilateral pyelograms were normal. The rest of the bladder appeared normal except for some moderate trabeculations throughout the bladder. There was enlargement of the lateral lobes of the prostate. The old TUR scar was visualized right at the bladder neck. Using diode side-firing fiber, the lateral lobes were taken down. The verumontanum, the external sphincter, and the ureteral openings were all intact at the end of the procedure. Pictures were taken and were shown to the family. At the end of the procedure, there was good hemostasis. A total of about 15 to 20 minutes of lasering time was used. A #22 3-way catheter was placed. At the end of the procedure, the patient was brought to recovery in stable condition. Plan was for removal of the Foley catheter in 48 hours and continuation of use of anticholinergics at night.   

KEYWORDS:urology, laser vaporization of the prostate, cystopyelogram, benign prostatic hypertroph, benign prostatic hypertrophy, alpha blockers, laser vaporization, anticholinergics, laser, vaporization, prostate, bladder

would be:

SAMPLE TYPE / MEDICAL SPECIALTY:Urology  
SAMPLE NAME: Laser Vaporization of Prostate 

DESCRIPTION:Cystopyelogram and laser vaporization of the prostate.  

PREOPERATIVE DIAGNOSIS:Benign prostatic hypertrophy.  

POSTOPERATIVE DIAGNOSIS:Benign prostatic hypertrophy.  

SURGERY: Cystopyelogram and laser vaporization of the prostate.  

ANESTHESIA:Spinal.  

ESTIMATED BLOOD LOSS:Minimal.  

FLUIDS:Crystalloid.  


BRIEF HISTORY:The patient is a 67-year-old male with a history of TURP, presented to us with urgency, frequency, and dribbling. The patient was started on alpha-blockers with some help, but had nocturia q.1h. The patient was given anticholinergics with minimal to no help. The patient had a cystoscopy done, which showed enlargement of the left lateral lobes of the prostate. At this point, options were discussed such as watchful waiting and laser vaporization to open up the prostate to get a better stream. Continuation of alpha-blockers and adding another anti-cholinergic at night to prevent bladder overactivity were discussed. The patient was told that his symptoms may be related to the mild-to-moderate trabeculation in the bladder, which can cause poor compliance. The patient understood and wanted to proceed with laser vaporization to see if it would help improve his stream, which in turn might help improve emptying of the bladder and might help his overactivity of the bladder. The patient was told that he may need anticholinergics. There could be increased risk of incontinence, stricture, erectile dysfunction, other complications and the consent was obtained.   

PROCEDURE IN DETAIL: The patient was brought to the OR and anesthesia was applied. The patient was placed in dorsal lithotomy position. The patient was given preoperative antibiotics. The patient was prepped and draped in the usual sterile fashion. A #23-French scope was inserted inside the urethra into the bladder under direct vision. Bilateral pyelograms were normal. The rest of the bladder appeared normal except for some moderate trabeculations throughout the bladder. There was enlargement of the lateral lobes of the prostate. The old TUR scar was visualized right at the bladder neck. Using diode side-firing fiber, the lateral lobes were taken down. The verumontanum, the external sphincter, and the ureteral openings were all intact at the end of the procedure. Pictures were taken and were shown to the family. At the end of the procedure, there was good hemostasis. A total of about 15 to 20 minutes of lasering time was used. A #22 3-way catheter was placed. At the end of the procedure, the patient was brought to recovery in stable condition. Plan was for removal of the Foley catheter in 48 hours and continuation of use of anticholinergics at night.   

KEYWORDS:urology, laser vaporization of the prostate, cystopyelogram, benign prostatic hypertroph, benign prostatic hypertrophy, alpha blockers, laser vaporization, anticholinergics, laser, vaporization, prostate, bladder

Answer 1

It's easier to achieve by using more than 1 regex substitution.

Example Code:

import re

test_str = ("SAMPLE TYPE / MEDICAL SPECIALTY:Urology  \n"
    "SAMPLE NAME: Laser Vaporization of Prostate \n\n"
    "DESCRIPTION:Cystopyelogram and laser vaporization of the prostate. \n"
    "(Medical Transcription Sample Report)  \n\n"
    "* * *\n\n"
    "Brief History: The patient is a 67-year-old male with a history of TURP, presented to us with urgency, frequency, and dribbling. \n\n"
    "The patient was started on alpha-blockers with some help, but had nocturia q.1h. The patient was given anticholinergics with minimal to no help. \n\n"
    "KEYWORDS:urology, laser vaporization of the prostate, cystopyelogram, benign prostatic hypertroph, benign prostatic hypertrophy, alpha blockers, laser vaporization, anticholinergics, laser, vaporization, prostate, bladder")

regex_selective_replace_newlines = r"\s*?\r?\n(?![A-Z].*?:)(?:[^\w\s].*)?"

result = re.sub(regex_selective_replace_newlines, ' ', test_str, 0)

regex_extra_newlines = r"(\r?\n)([A-Z].*?:)"

def subst_extra_upper(m):
  return m.group(1) + m.group(1) + m.group(2).upper()

result = re.sub(regex_extra_newlines, subst_extra_upper, result, 0)

regex_many_spaces = r"[ ]{3,}"
result = re.sub(regex_many_spaces, "  ", result, 0)

if result:
    print (result)

Output:

SAMPLE TYPE / MEDICAL SPECIALTY:Urology  

SAMPLE NAME: Laser Vaporization of Prostate 

DESCRIPTION:Cystopyelogram and laser vaporization of the prostate.  

BRIEF HISTORY: The patient is a 67-year-old male with a history of TURP, presented to us with urgency, frequency, and dribbling.  The patient was started on alpha-blockers with some help, but had nocturia q.1h. The patient was given anticholinergics with minimal to no help. 

KEYWORDS:urology, laser vaporization of the prostate, cystopyelogram, benign prostatic hypertroph, benign prostatic hypertrophy, alpha blockers, laser vaporization, anticholinergics, laser, vaporization, prostate, bladder

Remarks:

The first removes unneeded newlines.
And also lines that don't start with a letter or whitespace.

Then the 2nd regex adds extra newlines and upper's the titles.

And the 3rd regex simply replaces 3 or more spaces by 2 spaces.
Makes it look better.