linux命令输出一个csv文件（里面有不同的格式）

Question

command output :

//****** BMC SENSORS ******//
Object Id                     : 0xF000000
PCIe s:b:d.f                  : 0000:b2:00.0
Device Id                     : 0x0b30
Numa Node                     : 1
Ports Num                     : 01
Bitstream Id                  : 0x23000110030506
Bitstream Version             : 0.2.3
Pr Interface Id               : f3c99413-5081-4aad-bced-07eb84a6d0bb
( 1) Board Power              : 58.11 Watts
( 2) 12V Backplane Current    : 2.43 Amps
( 3) 12V Backplane Voltage    : 12.14 Volts
( 4) 1.2V Voltage             : 1.19 Volts
( 6) 1.8V Voltage             : 1.81 Volts
( 8) 3.3V Voltage             : 3.27 Volts
(10) FPGA Core Voltage        : 0.90 Volts
(11) FPGA Core Current        : 12.28 Amps
(12) FPGA Die Temperature     : 58.50 Celsius
(13) Board Temperature        : 43.00 Celsius
(14) QSFP0 Supply Voltage     : 0.00 Volts
(15) QSFP0 Temperature        : 0.00 Celsius
(24) 12V AUX Current          : 2.35 Amps
(25) 12V AUX Voltage          : 12.18 Volts
(37) QSFP1 Supply Voltage     : N/A
(38) QSFP1 Temperature        : N/A
(44) PKVL0 Core Temperature   : 0.00 Celsius
(45) PKVL0 SerDes Temperature : 0.00 Celsius
(46) PKVL1 Core Temperature   : 72.00 Celsius
(47) PKVL1 SerDes Temperature : 73.50 Celsius

I don't need first line, serial numbers. Want left side part as heading and right side part as column of each heading respectively in CSV.

I tried using sed in different ways -

sed ///****** BMC SENSORS ******/// fpgainfo bmc

fpgainfo bmc | sed ///****** BMC SENSORS ******/// >> ./fp.csv 

sed (1),(2),(3),(4),(6),(8),(10),(11),(12),(13),(14),(15),(24),(25),(37),(38),(44),(45),(46),(47)` fpgainfo bmc

but not ended up in what I expect. Kindly help me to achieve the same.

Answer 1

With the provided examples, would you please try the following:

fpgainfo bmc | sed -E '1d; s/^\([[:space:]]*[[:digit:]]+\)[[:space:]]*//; s/[[:space:]]+:[[:space:]]+/,/' > fp.csv

• 1d skips the 1st line.
• s/^\\([[:space:]]*[[:digit:]]+\\)[[:space:]]*// removes the leading serial numbers.
• s/[[:space:]]+:[[:space:]]+/,/ replaces the colon and the surrounding whitespaces with a comma.

Answer 2

Scroll down to the last paragraph for my best recommendation. This presents some background around your attempt just to explain what's wrong before coming to the conclusion.

Of your attempts, only the middle one does anything remotely useful or syntactically well-defined - but the sed syntax is still wrong; you need to use a different delimiter or backslash the literal slashes in the regex for it to work.

But sed is probably not a good tool for this; its processing model is basically constrained to examine one input line at a time. You can make it perform more challenging processing, but it's definitely not a good first exercise for a beginner.

I would perhaps prefer Awk for this. If you know another scripting language, or are planning to start learning one (eg Python) probably go with that instead; but Awk is straightforward enough that you can learn a good part of it in half an hour.

The surrounding shell script will look the same regardless; you'll want to run your command and pipe its output to something which knows how to transform the lines it receives on standard input into your desired format.

fpgainfo bmc | awk ...

Here, then, is a simple Awk script to extract (what I think you are trying to say are) the fields you want into a CSV file, with field names on a header line before the values.

awk 'BEGIN { sep=""; n=0 }
  /^\( *[1-9][0-9]*\)/ {
    # Parse out data; trim trailing whitespace
    key = substr($0, 6, 25)
    sub(/ +$/, "", key)
    value = substr($0, 33)
    sub(/ +$/, "", value)

    # Normalize index
    sub(/^\( +/, "")
    if ($1 ~ /^[123468]|1[12345]|2[45]|3[78]|4[4567])\)/) {
      printf("%s%s", sep, key)
      sep = ","
      v[++n] = value }
  }
  END { printf "\n"; sep=""; 
    for(i=1; i<=n; ++i) {
      printf("%s%s", sep, v[i])
      sep = "," }
    printf "\n"
  }'

Demo: https://ideone.com/BwRN72

But this is still brittle because it hinges on the assumption that the output you showed is completely representative of everything we need to handle. There could easily be a case where the program wraps a long line for legibility, or adjusts column widths depending on your terminal's size.

I'm not familiar with fpgainfo but most modern tools have an option to produce machine-readable output directly (JSON, YAML, XML etc); if that's available, it's generally vastly superior to hand-crafting a parser for mucky human-readable formats like this. Quick googling gets me https://opae.github.io/0.13.0/docs/fpga_tools/fpgainfo/fpgainfo.html which mentions a --json option - probably go with that instead (and notice that eg jq can further transform that into proper CSV with a simple option). It's not necessarily less work, but it's going to be hugely more robust because the formats are documented, well-specified, and designed for programmatic manipulation.

Answer 3

This might work for you (GNU sed):

sed -E '1{x;s/^/\n/;x};/^\(/!d
        G;s/^(....).*: (.*)\n(.*)\n(.*)/\3,\1\n\4,\2/;h;$!d;s/.(.*\n)./\1/' file

Prime the hold space with a newline (columns will be to the left of the newline and values to the right).

Junk the unwanted lines.

Append the hold space and substitute the current lines column and value into the accumulated columns/values.

Copy the accumulated columns/values to the hold space.

Delete all but the last line.

Remove comma artefacts and print the result.

---

Same solution but for real headings:

sed -E '1{x;s/^/\n/;x};/^\(/!d
        G;s/^.....(.*\S)\s*: (.*)\n(.*)\n(.*)/\3,\1\n\4,\2/;h;$!d;s/.(.*\n)./\1/' file

For heading with parens:

sed -E '1{x;s/^/\n/;x};/^\(/!d
        G;s/^(.*\S)\s*: (.*)\n(.*)\n(.*)/\3,\1\n\4,\2/;h;$!d;s/.(.*\n)./\1/' file

Answer 4

There are different ways to use awk to solve the problem.

awk -F ' *: *' 'NR>1 { sub(/^\([0-9 ]*\) */,"");
  if(heading) { heading=heading ","; data=data ","}
  heading=heading $1; data = data $2
}
END {print heading; print data}' inputfile

With the input shown in the question this prints

Object Id,PCIe s,Device Id,Numa Node,Ports Num,Bitstream Id,Bitstream Version,Pr Interface Id,Board Power,12V Backplane Current,12V Backplane Voltage,1.2V Voltage,1.8V Voltage,3.3V Voltage,FPGA Core Voltage,FPGA Core Current,FPGA Die Temperature,Board Temperature,QSFP0 Supply Voltage,QSFP0 Temperature,12V AUX Current,12V AUX Voltage,QSFP1 Supply Voltage,QSFP1 Temperature,PKVL0 Core Temperature,PKVL0 SerDes Temperature,PKVL1 Core Temperature,PKVL1 SerDes Temperature
0xF000000,b,0x0b30,1,01,0x23000110030506,0.2.3,f3c99413-5081-4aad-bced-07eb84a6d0bb,58.11 Watts,2.43 Amps,12.14 Volts,1.19 Volts,1.81 Volts,3.27 Volts,0.90 Volts,12.28 Amps,58.50 Celsius,43.00 Celsius,0.00 Volts,0.00 Celsius,2.35 Amps,12.18 Volts,N/A,N/A,0.00 Celsius,0.00 Celsius,72.00 Celsius,73.50 Celsius

Explanation:

• -F ' *: *' field separator is : with any number of leading and trailing spaces
• NR>1 do the commands in { ...} for all lines except first one
• sub(/^\\([0-9 ]*\\) */,""); any leading number in parentheses from the befinning of the line
• if(heading) { heading=heading ","; data=data ","} if(heading) { heading=heading ","; data=data ","} if heading is not empty, append a comma to both heading and data, ie us a comma as a delimiter except before the first values
• heading=heading $1; data = data $2 heading=heading $1; data = data $2 append the first field to the heading and the second to the data
• END {print heading; print data} END {print heading; print data} at the end print heading and data.

---

I tested this with GNU awk only, which should be the default version of awk on Linux.