Why does looping through an API create duplciate wordpress posts

Question

I'm trying to iterate over some API data and turn it into custom WordPress posts.

My function works, but I can't stop it from creating duplicate posts.

I am checking a unique unitproperty_id field on each iteration to avoid creating duplicates, but it only works in a sporadic way.

Sometimes there will be no duplicate posts, sometimes there will be two, and sometimes three.

It also only happens on the first pass, after the posts have been created (duplicates and non-duplicates) it won't keep creating them each time. The duplicates only appear on the first load.

When this function is working correctly I'll be calling it on a regular basis using wp_cron job, but until then I am calling it using the shutdown action hook that runs at the very end of WordPress loading.

Can you see why I this is generating duplicate posts?

add_action( 'shutdown', 'build_units_from_api', 10 );
function build_units_from_api() {

    //Get all the unit IDs currently inside wordpress
    $current_property_ids = array();
    $wp_units = get_all_wp_posts('unit');
    foreach ($wp_units as $wp_unit) {
        $current_property_ids[] = (int)get_post_meta( $wp_unit->ID, 'unitproperty_id', true );
    }

    //Get all the api data
    $property_units = get_all_property_units();
    $num_of_property_pages = $property_units['pages'];

    //Loop through the pages of the API data
    for ( $i=0; $i <= $num_of_property_pages; ++$i ) {

        $page_of_units = get_property_units_by_page($i);
        $num_of_units_on_page = count($page_of_units['results']);

        //Loop through the results of each page
        for ( $x=0; $x <= $num_of_units_on_page; ++$x ) {

            $property_profile = $page_of_units['results'][$x];

            //Check if we already have that unit property ID
            if ( in_array($property_profile['id'], $current_property_ids, true) || $property_profile['id'] == null ) {
                //Do nothing and exit the current iteration
                continue;
            } else {
                //Get the individual profile info from unit property API
                $api = new unitpropertyAPI();
                $api->getunit($property_profile['id']);
                $property_profile_data = json_decode($api->result, true);

                $post_arr = array(
                    'post_type' => 'unit',
                    'post_title' => $property_profile_data['name'],
                    'post_content' => '',
                    'post_status' => 'publish',
                    'comment_status' => 'closed',
                    'ping_status' => 'closed',
                    'meta_input' => array(
                        'unitproperty_id'   => $property_profile_data['id'],
                        'unit_name' => $property_profile_data['name'],                  
                        'unitproperty_rating'   => $property_profile_data['rating'],
                    ),
                );

                //Put those fields into a new 'unit' custom post
                $new_id = wp_insert_post( $post_arr );

                //Stop from adding this property again
                $current_property_ids[] = $property_profile_data['id'];
            }
        }
    }
}

Answer 1

It was the action hook!

I found a solution thanks to this answer: https://wordpress.stackexchange.com/questions/287599/wp-insert-post-creates-duplicates-with-post-status-publish

Changing the action hook to admin_notices fixes the problem because it won't re-fire during the wp_insert_post call.