其他两个字符串之间的grep字符串作为分隔符

Question

I have to do a report on how many times a certain CSS class appears in the content of our pages (over 10k pages). The trouble is, the header and footer contains that class, so a grep returns every single page.

So, how do I grep for content?

EDIT: I am looking for if a page has list-unstyled between <main> and </main>

So do I use a regular expression for that grep? or do I need to use PowerShell to have more functionality?

I have grep at my disposal and PowerShell, but I could use a portable software if that is my only option.

Ideally, I would get a report (.txt or .csv) with pages and line numbers where the class shows up, but just a list of the pages themselves would suffice.

EDIT: Progress

I now have this in PowerShell

$files = get-childitem -recurse -path w:\test\york\ -Filter *.html 
foreach ($file in $files)
{
$htmlfile=[System.IO.File]::ReadAllText($file.fullName)
$regex="(?m)<main([\w\W]*)</main>"
if ($htmlfile -match $regex) { 
    $middle=$matches[1] 
    [regex]::Matches($middle,"list-unstyled")
    Write-Host $file.fullName has matches in the middle:
}
}

Which I run with this command .\\FindStr.ps1 | Export-csv C:\\Tools\\text.csv .\\FindStr.ps1 | Export-csv C:\\Tools\\text.csv

it outputs the filename and path with string in the console, put does not add anything to the CSV. How can I get that added in?

Answer 1

Don't use string matches for something like this. Analyze the DOM instead. That should allow you to exclude headers and footers by selecting the appropriate root element.

$ie = New-Object -COM 'InternetExplorer.Application'

$url = '...'
$classname = 'list-unstyled'

$ie.Navigate($url)
do { Start-Sleep -Milliseconds 100 } until ($ie.ReadyState -eq 4)

$root = $ie.Document.getElementsById('content-element-id')
$hits = $root.getElementsByTagName('*') | ? { $_.ClassName -eq $classname }

$hits.Count  # number of occurrences of $classname below content element

Answer 2

You can create a regexp that will be suitable for multiline match. The regexp "(?m)<!-- main content -->([\\w\\W]*)<!-- end content -->" matches a multiline content delimited by your comments, with (?m) part meaning that this regexp has multiline option enabled. The group ([\\w\\W]*) matches everything between your comments, and also enables you to query $matches[1] which will contain your "main text" without headers and footers.

$htmlfile=[System.IO.File]::ReadAllText($fileToGrep)
$regex="(?m)<!-- main content -->([\w\W]*)<!-- end content -->"
if ($htmlfile -match $regex) { 
    $middle=$matches[1] 
    [regex]::Matches($middle,"list-unstyled")
}

This is only an example of how should you parse the file. You populate $fileToGrep with a file name which you desire to parse, then run this snippet to receive a string that contains all the list-unstyled strings in the middle of that file.

Answer 3

What Ansgar Wiechers' answer says is good advice. Don't string search html files. I don't have a problem with it but it is worth noting that not all html files are the same and regex searches can produce flawed results. If tools exists that are aware of the file content structure you should use them.

I would like to take a simple approach that reports all files that have enough occurrences of the text list-unstyled in all html files in a given directory. You expect there to be 2? So if more than that show up then there is enough. I would have done a more complicated regex solution but since you want the line number as well I came up with this compromise.

$pattern = "list-unstyled"
Get-ChildItem C:\temp -Recurse -Filter *.html | 
    Select-String $pattern | 
    Group-Object Path | 
    Where-Object{$_.Count -gt 2} | 
    ForEach-Object{
        $props = @{
            File = $_.Group | Select-Object -First 1 -ExpandProperty Path
            PatternFound = ($_.Group | Select-Object -ExpandProperty LineNumber) -join ";"
        }

        New-Object -TypeName PSCustomObject -Property $props
    }

Select-String is a grep like tool that can search files for string. It reports the located line number in the file which I why we are using it here.

You should get output that looks like this on your PowerShell console.

File                                                                           PatternFound                                                                  
----                                                                           ------------                                                                  
C:\temp\content.html                                                           4;11;54

Where 4,11,54 is the lines where the text was found. The code filters out results where the count of lines is less than 3. So if you expect it once in the header and footer those results should be excluded.