Need to split multiple files in a directory based on string, rename properly using powershell or fix my perl script

Question

I have a directory full of files (text exports of Dynamics NAV objects that have been exported) in Windows. Each file contains multiple objects. I need to split each file into separate files based on lines that begin with OBJECT, and name each file appropriately.

The purpose of this is to get our Dynamics NAV system into git.

I wrote a nifty perl program to do this that works great on linux. But it hangs on the while(<>) loop in Windows (Server 2012 if that matters).

So, I need to either figure out how to do this in the PowerShell script that I wrote that generates all of the files, or fix my perl script that I'm calling from PowerShell. Does Windows perl handle filehandles differently than linux?

Here's my code:

#!/usr/bin/perl

use strict;
use warnings;
use File::Path qw(make_path remove_tree);
use POSIX qw(strftime); 

my $username = getlogin || getpwuid($<);
my $datestamp  = strftime("%Y%m%d-%H%M%S", localtime); 

my $work_dir = "/temp/nav_export";
my $objects_dir = "$work_dir/$username/objects";
my $export_dir = "$work_dir/$username/$datestamp";

print "Objects being exported to $export_dir\n";

make_path("$export_dir/Page", "$export_dir/Codeunit",  "$export_dir/MenuSuite", "$export_dir/Query", "$export_dir/Report", "$export_dir/Table", "$export_dir/XMLport");

chdir $objects_dir or die "Could not change to $objects_dir: $!";

# delete empty files
foreach(glob('*.*')) {
    unlink if -f and !-s _;
}

my @files = <*>;
my $count = @files;
print "Processing $count files\n";

open (my $fh, ">-") or die "Could not open standard out: $!";

# OBJECT Codeunit 1 ApplicationManagement

while(<>)
{
    if (m/^OBJECT ([A-Za-z]+) ([0-9]+) (.*)/o)
    {
        my $objectType = $1;
        my $objectID = $2;
        my $objectName = my $firstLine = $3;
        $objectName =~ s/[\. \/\(\)\\]/_/g; # translate spaces, (, ), ., \ and / to underscores
        $objectName =~ tr/\cM//d; # get rid of Ctrl-M
        my $filename = $export_dir . "/" . $objectType . "/" . $objectType . "~" . $objectID . "~" . $objectName;

        close $fh and open($fh, '>', $filename) or die "Could not open file '$filename' $!";

        print $fh "OBJECT $objectType $objectID $firstLine\n";

        next;
    }

    print $fh $_;
}

I've learned quite a bit of PowerShell in the past few days. There are some things that it really does quite well. And some (such as calling an executable with variables and command line options that have spaces) that are maddeningly difficult to figure out. To call curl, this is what I resorted to:

$curl = "C:\Program Files (x86)\cURL\bin\curl"

$arg10 = '-s'
$arg1 = '-X'
$arg11 = 'post'
$arg2 = '-H'
$arg22 = '"Accept-Encoding: gzip,deflate"'
$arg3 = '-H'
$arg33 = '"Content-Type: text/xml;charset=UTF-8"'
$arg4 = '-H'
$arg44 = '"SOAPAction:urn:microsoft-dynamics-schemas/page/permissionrange:ReadMultiple"'
$arg5 = '--ntlm'
$arg6 = '-u'
$arg66 = 'username:password'
$arg7 = '-d'
$arg77 = '"@soap_envelope.txt"'
$arg8 =  "http://$servicetier.corp.company.net:7047/$database/WS/DBDOC/Page/PermissionRange"
$arg9 = "-o"
$arg99 = "c:\temp\nav_export\$env:username\raw_list.xml"

&"$curl" $arg10 $arg1 $arg11 $arg2 $arg22 $arg3 $arg33 $arg4 $arg44 $arg5 $arg6 $arg66 $arg7 $arg77 $arg8 $arg9 $arg99

I realize that part is a bit of a tangent. But I've been working really hard at trying to figure this out and not have to bother you nice folk here at stackoverflow!

I'm ambivalent about making it work in PowerShell or fixing the Perl code at this point. I just need to make it work. But I'm hoping it's just some little difference in filehandle handling between linux and Windows.

Answer 1

It's hard to believe that the Perl code that you show does anything on Linux either. It looks like your while loop is supposed to be reading through all of the files in the @files array, but to make it do that you have to copy the names to @ARGV .

Also note that @files will contain directories as well as files.

I suggest you change the lines starting with my @files = <*> to this. There's no reason why it shouldn't work on both Windows and Linux.

our @ARGV = grep -f, glob '*';
my $count = @ARGV;
print "Processing $count files\n";

my $fh;

while (<>) {

    s/\s+\z//;  # Remove trailing whitespace (including CR and LF)
    my @fields = split ' ', $_, 4;

    if ( @fields == 4 and $fields[0] eq 'OBJECT' ) {

        my ($object_type, $object_id, $object_name) = @fields[1,2,3];
        $object_name =~ tr{ ().\\/}{_}; # translate spaces, (, ), ., \ and / to underscores

        my $filename = "$export_dir/$object_type/$object_type~$object_id~$object_name";

        open $fh, '>', $filename or die "Could not open file '$filename': $!";
    }

    print $fh "$_\n" if $fh;

    if (eof) {
        close $fh;
        $fh = undef;
    }
}