How to split a single file into multiple files based on a column in linux?












0















I have a text file with following information:



Hugo_Symbol Tumor_Sample_Barcode    Entrez_Gene_Id  Center  NCBI_Build

MTHFR   TCGA-BD-A2L6-01A-11D-A20W-10    4524    BCM GRCh38

SLC30A1 TCGA-BD-A2L6-01A-11D-A20W-10    7779    BCM GRCh38

USH2A   TCGA-BD-A2L6-01A-11D-A20W-10    7399    BCM GRCh38

SOS1    TCGA-BD-A2L6-01A-11D-A20W-10    6654    BCM GRCh38

TMEM51  TCGA-O8-A75V-01A-11D-A32G-10    55092   BCM GRCh38

FLG TCGA-O8-A75V-01A-11D-A32G-10    2312    BCM GRCh38

FLG TCGA-O8-A75V-01A-11D-A32G-10    2312    BCM GRCh38

PRDM16  TCGA-G3-A7M5-01A-11D-A33Q-10    63976   BCM GRCh38

DNAJC11 TCGA-G3-A7M5-01A-11D-A33Q-10    55735   BCM GRCh38

HNRNPCL2    TCGA-G3-A7M5-01A-11D-A33Q-10    440563  BCM GRCh38

C1orf94 TCGA-G3-A7M5-01A-11D-A33Q-10    84970   BCM GRCh38

NFYC    TCGA-G3-A7M5-01A-11D-A33Q-10    4802    BCM GRCh38

IPP TCGA-G3-A7M5-01A-11D-A33Q-10    3652    BCM GRCh38


As you see there are multiple samples I want to split the file into multiple files based on the column "Tumor_Sample_Barcode". The output files need to be named with samplename.txt.



First output - TCGA-BD-A2L6-01A-11D-A20W-10.txt



Hugo_Symbol Tumor_Sample_Barcode    Entrez_Gene_Id  Center  NCBI_Build

MTHFR   TCGA-BD-A2L6-01A-11D-A20W-10    4524    BCM GRCh38

SLC30A1 TCGA-BD-A2L6-01A-11D-A20W-10    7779    BCM GRCh38

USH2A   TCGA-BD-A2L6-01A-11D-A20W-10    7399    BCM GRCh38

SOS1    TCGA-BD-A2L6-01A-11D-A20W-10    6654    BCM GRCh38


Second output - TCGA-O8-A75V-01A-11D-A32G-10.txt



Hugo_Symbol Tumor_Sample_Barcode    Entrez_Gene_Id  Center  NCBI_Build

TMEM51  TCGA-O8-A75V-01A-11D-A32G-10    55092   BCM GRCh38

FLG TCGA-O8-A75V-01A-11D-A32G-10    2312    BCM GRCh38

FLG TCGA-O8-A75V-01A-11D-A32G-10    2312    BCM GRCh38


Third output - TCGA-G3-A7M5-01A-11D-A33Q-10.txt



Hugo_Symbol Tumor_Sample_Barcode    Entrez_Gene_Id  Center  NCBI_Build

PRDM16  TCGA-G3-A7M5-01A-11D-A33Q-10    63976   BCM GRCh38

DNAJC11 TCGA-G3-A7M5-01A-11D-A33Q-10    55735   BCM GRCh38

HNRNPCL2    TCGA-G3-A7M5-01A-11D-A33Q-10    440563  BCM GRCh38

C1orf94 TCGA-G3-A7M5-01A-11D-A33Q-10    84970   BCM GRCh38

NFYC    TCGA-G3-A7M5-01A-11D-A33Q-10    4802    BCM GRCh38

IPP TCGA-G3-A7M5-01A-11D-A33Q-10    3652    BCM GRCh38


How to do this linux?






linux files split







share|improve this question

























  • Thank you for the reply. But I don't see any headers in the output files. How to get the columns names also in the outputs?

    – user3351523
    Jan 31 '18 at 12:51






  • 1





    @user3351523, "headers" should be the next moment. The first moment should be your posting a testable input (as a text, not as image)

    – RomanPerekhrest
    Jan 31 '18 at 12:53













  • Yes, sorry for that. I posted test table input as text now. How to get the headers in output files?

    – user3351523
    Jan 31 '18 at 12:57
















0















I have a text file with following information:



Hugo_Symbol Tumor_Sample_Barcode    Entrez_Gene_Id  Center  NCBI_Build

MTHFR   TCGA-BD-A2L6-01A-11D-A20W-10    4524    BCM GRCh38

SLC30A1 TCGA-BD-A2L6-01A-11D-A20W-10    7779    BCM GRCh38

USH2A   TCGA-BD-A2L6-01A-11D-A20W-10    7399    BCM GRCh38

SOS1    TCGA-BD-A2L6-01A-11D-A20W-10    6654    BCM GRCh38

TMEM51  TCGA-O8-A75V-01A-11D-A32G-10    55092   BCM GRCh38

FLG TCGA-O8-A75V-01A-11D-A32G-10    2312    BCM GRCh38

FLG TCGA-O8-A75V-01A-11D-A32G-10    2312    BCM GRCh38

PRDM16  TCGA-G3-A7M5-01A-11D-A33Q-10    63976   BCM GRCh38

DNAJC11 TCGA-G3-A7M5-01A-11D-A33Q-10    55735   BCM GRCh38

HNRNPCL2    TCGA-G3-A7M5-01A-11D-A33Q-10    440563  BCM GRCh38

C1orf94 TCGA-G3-A7M5-01A-11D-A33Q-10    84970   BCM GRCh38

NFYC    TCGA-G3-A7M5-01A-11D-A33Q-10    4802    BCM GRCh38

IPP TCGA-G3-A7M5-01A-11D-A33Q-10    3652    BCM GRCh38


As you see there are multiple samples I want to split the file into multiple files based on the column "Tumor_Sample_Barcode". The output files need to be named with samplename.txt.



First output - TCGA-BD-A2L6-01A-11D-A20W-10.txt



Hugo_Symbol Tumor_Sample_Barcode    Entrez_Gene_Id  Center  NCBI_Build

MTHFR   TCGA-BD-A2L6-01A-11D-A20W-10    4524    BCM GRCh38

SLC30A1 TCGA-BD-A2L6-01A-11D-A20W-10    7779    BCM GRCh38

USH2A   TCGA-BD-A2L6-01A-11D-A20W-10    7399    BCM GRCh38

SOS1    TCGA-BD-A2L6-01A-11D-A20W-10    6654    BCM GRCh38


Second output - TCGA-O8-A75V-01A-11D-A32G-10.txt



Hugo_Symbol Tumor_Sample_Barcode    Entrez_Gene_Id  Center  NCBI_Build

TMEM51  TCGA-O8-A75V-01A-11D-A32G-10    55092   BCM GRCh38

FLG TCGA-O8-A75V-01A-11D-A32G-10    2312    BCM GRCh38

FLG TCGA-O8-A75V-01A-11D-A32G-10    2312    BCM GRCh38


Third output - TCGA-G3-A7M5-01A-11D-A33Q-10.txt



Hugo_Symbol Tumor_Sample_Barcode    Entrez_Gene_Id  Center  NCBI_Build

PRDM16  TCGA-G3-A7M5-01A-11D-A33Q-10    63976   BCM GRCh38

DNAJC11 TCGA-G3-A7M5-01A-11D-A33Q-10    55735   BCM GRCh38

HNRNPCL2    TCGA-G3-A7M5-01A-11D-A33Q-10    440563  BCM GRCh38

C1orf94 TCGA-G3-A7M5-01A-11D-A33Q-10    84970   BCM GRCh38

NFYC    TCGA-G3-A7M5-01A-11D-A33Q-10    4802    BCM GRCh38

IPP TCGA-G3-A7M5-01A-11D-A33Q-10    3652    BCM GRCh38


How to do this linux?






linux files split







share|improve this question

























  • Thank you for the reply. But I don't see any headers in the output files. How to get the columns names also in the outputs?

    – user3351523
    Jan 31 '18 at 12:51






  • 1





    @user3351523, "headers" should be the next moment. The first moment should be your posting a testable input (as a text, not as image)

    – RomanPerekhrest
    Jan 31 '18 at 12:53













  • Yes, sorry for that. I posted test table input as text now. How to get the headers in output files?

    – user3351523
    Jan 31 '18 at 12:57














0












0








0


1






I have a text file with following information:



Hugo_Symbol Tumor_Sample_Barcode    Entrez_Gene_Id  Center  NCBI_Build

MTHFR   TCGA-BD-A2L6-01A-11D-A20W-10    4524    BCM GRCh38

SLC30A1 TCGA-BD-A2L6-01A-11D-A20W-10    7779    BCM GRCh38

USH2A   TCGA-BD-A2L6-01A-11D-A20W-10    7399    BCM GRCh38

SOS1    TCGA-BD-A2L6-01A-11D-A20W-10    6654    BCM GRCh38

TMEM51  TCGA-O8-A75V-01A-11D-A32G-10    55092   BCM GRCh38

FLG TCGA-O8-A75V-01A-11D-A32G-10    2312    BCM GRCh38

FLG TCGA-O8-A75V-01A-11D-A32G-10    2312    BCM GRCh38

PRDM16  TCGA-G3-A7M5-01A-11D-A33Q-10    63976   BCM GRCh38

DNAJC11 TCGA-G3-A7M5-01A-11D-A33Q-10    55735   BCM GRCh38

HNRNPCL2    TCGA-G3-A7M5-01A-11D-A33Q-10    440563  BCM GRCh38

C1orf94 TCGA-G3-A7M5-01A-11D-A33Q-10    84970   BCM GRCh38

NFYC    TCGA-G3-A7M5-01A-11D-A33Q-10    4802    BCM GRCh38

IPP TCGA-G3-A7M5-01A-11D-A33Q-10    3652    BCM GRCh38


As you see there are multiple samples I want to split the file into multiple files based on the column "Tumor_Sample_Barcode". The output files need to be named with samplename.txt.



First output - TCGA-BD-A2L6-01A-11D-A20W-10.txt



Hugo_Symbol Tumor_Sample_Barcode    Entrez_Gene_Id  Center  NCBI_Build

MTHFR   TCGA-BD-A2L6-01A-11D-A20W-10    4524    BCM GRCh38

SLC30A1 TCGA-BD-A2L6-01A-11D-A20W-10    7779    BCM GRCh38

USH2A   TCGA-BD-A2L6-01A-11D-A20W-10    7399    BCM GRCh38

SOS1    TCGA-BD-A2L6-01A-11D-A20W-10    6654    BCM GRCh38


Second output - TCGA-O8-A75V-01A-11D-A32G-10.txt



Hugo_Symbol Tumor_Sample_Barcode    Entrez_Gene_Id  Center  NCBI_Build

TMEM51  TCGA-O8-A75V-01A-11D-A32G-10    55092   BCM GRCh38

FLG TCGA-O8-A75V-01A-11D-A32G-10    2312    BCM GRCh38

FLG TCGA-O8-A75V-01A-11D-A32G-10    2312    BCM GRCh38


Third output - TCGA-G3-A7M5-01A-11D-A33Q-10.txt



Hugo_Symbol Tumor_Sample_Barcode    Entrez_Gene_Id  Center  NCBI_Build

PRDM16  TCGA-G3-A7M5-01A-11D-A33Q-10    63976   BCM GRCh38

DNAJC11 TCGA-G3-A7M5-01A-11D-A33Q-10    55735   BCM GRCh38

HNRNPCL2    TCGA-G3-A7M5-01A-11D-A33Q-10    440563  BCM GRCh38

C1orf94 TCGA-G3-A7M5-01A-11D-A33Q-10    84970   BCM GRCh38

NFYC    TCGA-G3-A7M5-01A-11D-A33Q-10    4802    BCM GRCh38

IPP TCGA-G3-A7M5-01A-11D-A33Q-10    3652    BCM GRCh38


How to do this linux?






linux files split







share|improve this question
















I have a text file with following information:



Hugo_Symbol Tumor_Sample_Barcode    Entrez_Gene_Id  Center  NCBI_Build

MTHFR   TCGA-BD-A2L6-01A-11D-A20W-10    4524    BCM GRCh38

SLC30A1 TCGA-BD-A2L6-01A-11D-A20W-10    7779    BCM GRCh38

USH2A   TCGA-BD-A2L6-01A-11D-A20W-10    7399    BCM GRCh38

SOS1    TCGA-BD-A2L6-01A-11D-A20W-10    6654    BCM GRCh38

TMEM51  TCGA-O8-A75V-01A-11D-A32G-10    55092   BCM GRCh38

FLG TCGA-O8-A75V-01A-11D-A32G-10    2312    BCM GRCh38

FLG TCGA-O8-A75V-01A-11D-A32G-10    2312    BCM GRCh38

PRDM16  TCGA-G3-A7M5-01A-11D-A33Q-10    63976   BCM GRCh38

DNAJC11 TCGA-G3-A7M5-01A-11D-A33Q-10    55735   BCM GRCh38

HNRNPCL2    TCGA-G3-A7M5-01A-11D-A33Q-10    440563  BCM GRCh38

C1orf94 TCGA-G3-A7M5-01A-11D-A33Q-10    84970   BCM GRCh38

NFYC    TCGA-G3-A7M5-01A-11D-A33Q-10    4802    BCM GRCh38

IPP TCGA-G3-A7M5-01A-11D-A33Q-10    3652    BCM GRCh38


As you see there are multiple samples I want to split the file into multiple files based on the column "Tumor_Sample_Barcode". The output files need to be named with samplename.txt.



First output - TCGA-BD-A2L6-01A-11D-A20W-10.txt



Hugo_Symbol Tumor_Sample_Barcode    Entrez_Gene_Id  Center  NCBI_Build

MTHFR   TCGA-BD-A2L6-01A-11D-A20W-10    4524    BCM GRCh38

SLC30A1 TCGA-BD-A2L6-01A-11D-A20W-10    7779    BCM GRCh38

USH2A   TCGA-BD-A2L6-01A-11D-A20W-10    7399    BCM GRCh38

SOS1    TCGA-BD-A2L6-01A-11D-A20W-10    6654    BCM GRCh38


Second output - TCGA-O8-A75V-01A-11D-A32G-10.txt



Hugo_Symbol Tumor_Sample_Barcode    Entrez_Gene_Id  Center  NCBI_Build

TMEM51  TCGA-O8-A75V-01A-11D-A32G-10    55092   BCM GRCh38

FLG TCGA-O8-A75V-01A-11D-A32G-10    2312    BCM GRCh38

FLG TCGA-O8-A75V-01A-11D-A32G-10    2312    BCM GRCh38


Third output - TCGA-G3-A7M5-01A-11D-A33Q-10.txt



Hugo_Symbol Tumor_Sample_Barcode    Entrez_Gene_Id  Center  NCBI_Build

PRDM16  TCGA-G3-A7M5-01A-11D-A33Q-10    63976   BCM GRCh38

DNAJC11 TCGA-G3-A7M5-01A-11D-A33Q-10    55735   BCM GRCh38

HNRNPCL2    TCGA-G3-A7M5-01A-11D-A33Q-10    440563  BCM GRCh38

C1orf94 TCGA-G3-A7M5-01A-11D-A33Q-10    84970   BCM GRCh38

NFYC    TCGA-G3-A7M5-01A-11D-A33Q-10    4802    BCM GRCh38

IPP TCGA-G3-A7M5-01A-11D-A33Q-10    3652    BCM GRCh38


How to do this linux?






linux files split




linux files split






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Jan 31 '18 at 12:56







user3351523

















asked Jan 31 '18 at 12:41









user3351523user3351523

15739




15739













  • Thank you for the reply. But I don't see any headers in the output files. How to get the columns names also in the outputs?

    – user3351523
    Jan 31 '18 at 12:51






  • 1





    @user3351523, "headers" should be the next moment. The first moment should be your posting a testable input (as a text, not as image)

    – RomanPerekhrest
    Jan 31 '18 at 12:53













  • Yes, sorry for that. I posted test table input as text now. How to get the headers in output files?

    – user3351523
    Jan 31 '18 at 12:57



















  • Thank you for the reply. But I don't see any headers in the output files. How to get the columns names also in the outputs?

    – user3351523
    Jan 31 '18 at 12:51






  • 1





    @user3351523, "headers" should be the next moment. The first moment should be your posting a testable input (as a text, not as image)

    – RomanPerekhrest
    Jan 31 '18 at 12:53













  • Yes, sorry for that. I posted test table input as text now. How to get the headers in output files?

    – user3351523
    Jan 31 '18 at 12:57

















Thank you for the reply. But I don't see any headers in the output files. How to get the columns names also in the outputs?

– user3351523
Jan 31 '18 at 12:51





Thank you for the reply. But I don't see any headers in the output files. How to get the columns names also in the outputs?

– user3351523
Jan 31 '18 at 12:51




1




1





@user3351523, "headers" should be the next moment. The first moment should be your posting a testable input (as a text, not as image)

– RomanPerekhrest
Jan 31 '18 at 12:53







@user3351523, "headers" should be the next moment. The first moment should be your posting a testable input (as a text, not as image)

– RomanPerekhrest
Jan 31 '18 at 12:53















Yes, sorry for that. I posted test table input as text now. How to get the headers in output files?

– user3351523
Jan 31 '18 at 12:57





Yes, sorry for that. I posted test table input as text now. How to get the headers in output files?

– user3351523
Jan 31 '18 at 12:57










1 Answer
1






active

oldest

votes


















3














Awk solution:



awk 'NR==1{ h=$0 }NR>1{ print (!a[$2]++? h ORS $0 : $0) > $2".txt" }' file




  • NR==1{ h=$0 } - capture the 1st line/record as header line (NR points to a record number, $0 - contains the current line)


  • NR > 1 - for all records except the first one:



    • <cond>? <operand_1> : <operand_2> - classical ternary operator


    • !a[$2]++? - check for the 1st occurrence of barcode value $2 used as a key of associative array a


    • h ORS $0 - common header line concatenated with ORS(output record separator, defaults to n) and current record $0


    • print ... > $2".txt" - print custom content or the current line(if nothing was specified) into file <barcode_value>.txt





Or a more self-explanatory version:



awk 'NR==1 {header = $0; next}

     !header_printed[$2]++ {print header > $2".txt"}

     {print > $2".txt"}' < file


Viewing results:



$ head TCGA*.txt

==> TCGA-BD-A2L6-01A-11D-A20W-10.txt <==

Hugo_Symbol Tumor_Sample_Barcode    Entrez_Gene_Id  Center  NCBI_Build

MTHFR   TCGA-BD-A2L6-01A-11D-A20W-10    4524    BCM GRCh38

SLC30A1 TCGA-BD-A2L6-01A-11D-A20W-10    7779    BCM GRCh38

USH2A   TCGA-BD-A2L6-01A-11D-A20W-10    7399    BCM GRCh38

SOS1    TCGA-BD-A2L6-01A-11D-A20W-10    6654    BCM GRCh38



==> TCGA-G3-A7M5-01A-11D-A33Q-10.txt <==

Hugo_Symbol Tumor_Sample_Barcode    Entrez_Gene_Id  Center  NCBI_Build

PRDM16  TCGA-G3-A7M5-01A-11D-A33Q-10    63976   BCM GRCh38

DNAJC11 TCGA-G3-A7M5-01A-11D-A33Q-10    55735   BCM GRCh38

HNRNPCL2    TCGA-G3-A7M5-01A-11D-A33Q-10    440563  BCM GRCh38

C1orf94 TCGA-G3-A7M5-01A-11D-A33Q-10    84970   BCM GRCh38

NFYC    TCGA-G3-A7M5-01A-11D-A33Q-10    4802    BCM GRCh38

IPP TCGA-G3-A7M5-01A-11D-A33Q-10    3652    BCM GRCh38



==> TCGA-O8-A75V-01A-11D-A32G-10.txt <==

Hugo_Symbol Tumor_Sample_Barcode    Entrez_Gene_Id  Center  NCBI_Build

TMEM51  TCGA-O8-A75V-01A-11D-A32G-10    55092   BCM GRCh38

FLG TCGA-O8-A75V-01A-11D-A32G-10    2312    BCM GRCh38

FLG TCGA-O8-A75V-01A-11D-A32G-10    2312    BCM GRCh38



To adjust a filename based on 15-char sequence of barcode value:



awk 'NR==1{ h=$0 }NR>1{ print (!a[$2]++? h ORS $0 : $0) > substr($2, 1, 15)".txt" }' file





share|improve this answer


























  • Thank you very much !! Could you please explain the command?

    – user3351523
    Jan 31 '18 at 13:13













  • Please explain the command and could you also tell me how to get the output files with only 0-15 substring in the sample names like TCGA-BD-A2L6-01.txt, TCGA-G3-A7M5-01.txt and TCGA-O8-A75V-01.txt

    – user3351523
    Jan 31 '18 at 13:31






  • 1





    @user3351523, yes, see my explanation

    – RomanPerekhrest
    Jan 31 '18 at 13:47











Your Answer








StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














draft saved

draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f420938%2fhow-to-split-a-single-file-into-multiple-files-based-on-a-column-in-linux%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown

























1 Answer
1






active

oldest

votes








1 Answer
1






active

oldest

votes









active

oldest

votes






active

oldest

votes









3














Awk solution:



awk 'NR==1{ h=$0 }NR>1{ print (!a[$2]++? h ORS $0 : $0) > $2".txt" }' file




  • NR==1{ h=$0 } - capture the 1st line/record as header line (NR points to a record number, $0 - contains the current line)


  • NR > 1 - for all records except the first one:



    • <cond>? <operand_1> : <operand_2> - classical ternary operator


    • !a[$2]++? - check for the 1st occurrence of barcode value $2 used as a key of associative array a


    • h ORS $0 - common header line concatenated with ORS(output record separator, defaults to n) and current record $0


    • print ... > $2".txt" - print custom content or the current line(if nothing was specified) into file <barcode_value>.txt





Or a more self-explanatory version:



awk 'NR==1 {header = $0; next}

     !header_printed[$2]++ {print header > $2".txt"}

     {print > $2".txt"}' < file


Viewing results:



$ head TCGA*.txt

==> TCGA-BD-A2L6-01A-11D-A20W-10.txt <==

Hugo_Symbol Tumor_Sample_Barcode    Entrez_Gene_Id  Center  NCBI_Build

MTHFR   TCGA-BD-A2L6-01A-11D-A20W-10    4524    BCM GRCh38

SLC30A1 TCGA-BD-A2L6-01A-11D-A20W-10    7779    BCM GRCh38

USH2A   TCGA-BD-A2L6-01A-11D-A20W-10    7399    BCM GRCh38

SOS1    TCGA-BD-A2L6-01A-11D-A20W-10    6654    BCM GRCh38



==> TCGA-G3-A7M5-01A-11D-A33Q-10.txt <==

Hugo_Symbol Tumor_Sample_Barcode    Entrez_Gene_Id  Center  NCBI_Build

PRDM16  TCGA-G3-A7M5-01A-11D-A33Q-10    63976   BCM GRCh38

DNAJC11 TCGA-G3-A7M5-01A-11D-A33Q-10    55735   BCM GRCh38

HNRNPCL2    TCGA-G3-A7M5-01A-11D-A33Q-10    440563  BCM GRCh38

C1orf94 TCGA-G3-A7M5-01A-11D-A33Q-10    84970   BCM GRCh38

NFYC    TCGA-G3-A7M5-01A-11D-A33Q-10    4802    BCM GRCh38

IPP TCGA-G3-A7M5-01A-11D-A33Q-10    3652    BCM GRCh38



==> TCGA-O8-A75V-01A-11D-A32G-10.txt <==

Hugo_Symbol Tumor_Sample_Barcode    Entrez_Gene_Id  Center  NCBI_Build

TMEM51  TCGA-O8-A75V-01A-11D-A32G-10    55092   BCM GRCh38

FLG TCGA-O8-A75V-01A-11D-A32G-10    2312    BCM GRCh38

FLG TCGA-O8-A75V-01A-11D-A32G-10    2312    BCM GRCh38



To adjust a filename based on 15-char sequence of barcode value:



awk 'NR==1{ h=$0 }NR>1{ print (!a[$2]++? h ORS $0 : $0) > substr($2, 1, 15)".txt" }' file





share|improve this answer


























  • Thank you very much !! Could you please explain the command?

    – user3351523
    Jan 31 '18 at 13:13













  • Please explain the command and could you also tell me how to get the output files with only 0-15 substring in the sample names like TCGA-BD-A2L6-01.txt, TCGA-G3-A7M5-01.txt and TCGA-O8-A75V-01.txt

    – user3351523
    Jan 31 '18 at 13:31






  • 1





    @user3351523, yes, see my explanation

    – RomanPerekhrest
    Jan 31 '18 at 13:47
















3














Awk solution:



awk 'NR==1{ h=$0 }NR>1{ print (!a[$2]++? h ORS $0 : $0) > $2".txt" }' file




  • NR==1{ h=$0 } - capture the 1st line/record as header line (NR points to a record number, $0 - contains the current line)


  • NR > 1 - for all records except the first one:



    • <cond>? <operand_1> : <operand_2> - classical ternary operator


    • !a[$2]++? - check for the 1st occurrence of barcode value $2 used as a key of associative array a


    • h ORS $0 - common header line concatenated with ORS(output record separator, defaults to n) and current record $0


    • print ... > $2".txt" - print custom content or the current line(if nothing was specified) into file <barcode_value>.txt





Or a more self-explanatory version:



awk 'NR==1 {header = $0; next}

     !header_printed[$2]++ {print header > $2".txt"}

     {print > $2".txt"}' < file


Viewing results:



$ head TCGA*.txt

==> TCGA-BD-A2L6-01A-11D-A20W-10.txt <==

Hugo_Symbol Tumor_Sample_Barcode    Entrez_Gene_Id  Center  NCBI_Build

MTHFR   TCGA-BD-A2L6-01A-11D-A20W-10    4524    BCM GRCh38

SLC30A1 TCGA-BD-A2L6-01A-11D-A20W-10    7779    BCM GRCh38

USH2A   TCGA-BD-A2L6-01A-11D-A20W-10    7399    BCM GRCh38

SOS1    TCGA-BD-A2L6-01A-11D-A20W-10    6654    BCM GRCh38



==> TCGA-G3-A7M5-01A-11D-A33Q-10.txt <==

Hugo_Symbol Tumor_Sample_Barcode    Entrez_Gene_Id  Center  NCBI_Build

PRDM16  TCGA-G3-A7M5-01A-11D-A33Q-10    63976   BCM GRCh38

DNAJC11 TCGA-G3-A7M5-01A-11D-A33Q-10    55735   BCM GRCh38

HNRNPCL2    TCGA-G3-A7M5-01A-11D-A33Q-10    440563  BCM GRCh38

C1orf94 TCGA-G3-A7M5-01A-11D-A33Q-10    84970   BCM GRCh38

NFYC    TCGA-G3-A7M5-01A-11D-A33Q-10    4802    BCM GRCh38

IPP TCGA-G3-A7M5-01A-11D-A33Q-10    3652    BCM GRCh38



==> TCGA-O8-A75V-01A-11D-A32G-10.txt <==

Hugo_Symbol Tumor_Sample_Barcode    Entrez_Gene_Id  Center  NCBI_Build

TMEM51  TCGA-O8-A75V-01A-11D-A32G-10    55092   BCM GRCh38

FLG TCGA-O8-A75V-01A-11D-A32G-10    2312    BCM GRCh38

FLG TCGA-O8-A75V-01A-11D-A32G-10    2312    BCM GRCh38



To adjust a filename based on 15-char sequence of barcode value:



awk 'NR==1{ h=$0 }NR>1{ print (!a[$2]++? h ORS $0 : $0) > substr($2, 1, 15)".txt" }' file





share|improve this answer


























  • Thank you very much !! Could you please explain the command?

    – user3351523
    Jan 31 '18 at 13:13













  • Please explain the command and could you also tell me how to get the output files with only 0-15 substring in the sample names like TCGA-BD-A2L6-01.txt, TCGA-G3-A7M5-01.txt and TCGA-O8-A75V-01.txt

    – user3351523
    Jan 31 '18 at 13:31






  • 1





    @user3351523, yes, see my explanation

    – RomanPerekhrest
    Jan 31 '18 at 13:47














3












3








3







Awk solution:



awk 'NR==1{ h=$0 }NR>1{ print (!a[$2]++? h ORS $0 : $0) > $2".txt" }' file




  • NR==1{ h=$0 } - capture the 1st line/record as header line (NR points to a record number, $0 - contains the current line)


  • NR > 1 - for all records except the first one:



    • <cond>? <operand_1> : <operand_2> - classical ternary operator


    • !a[$2]++? - check for the 1st occurrence of barcode value $2 used as a key of associative array a


    • h ORS $0 - common header line concatenated with ORS(output record separator, defaults to n) and current record $0


    • print ... > $2".txt" - print custom content or the current line(if nothing was specified) into file <barcode_value>.txt





Or a more self-explanatory version:



awk 'NR==1 {header = $0; next}

     !header_printed[$2]++ {print header > $2".txt"}

     {print > $2".txt"}' < file


Viewing results:



$ head TCGA*.txt

==> TCGA-BD-A2L6-01A-11D-A20W-10.txt <==

Hugo_Symbol Tumor_Sample_Barcode    Entrez_Gene_Id  Center  NCBI_Build

MTHFR   TCGA-BD-A2L6-01A-11D-A20W-10    4524    BCM GRCh38

SLC30A1 TCGA-BD-A2L6-01A-11D-A20W-10    7779    BCM GRCh38

USH2A   TCGA-BD-A2L6-01A-11D-A20W-10    7399    BCM GRCh38

SOS1    TCGA-BD-A2L6-01A-11D-A20W-10    6654    BCM GRCh38



==> TCGA-G3-A7M5-01A-11D-A33Q-10.txt <==

Hugo_Symbol Tumor_Sample_Barcode    Entrez_Gene_Id  Center  NCBI_Build

PRDM16  TCGA-G3-A7M5-01A-11D-A33Q-10    63976   BCM GRCh38

DNAJC11 TCGA-G3-A7M5-01A-11D-A33Q-10    55735   BCM GRCh38

HNRNPCL2    TCGA-G3-A7M5-01A-11D-A33Q-10    440563  BCM GRCh38

C1orf94 TCGA-G3-A7M5-01A-11D-A33Q-10    84970   BCM GRCh38

NFYC    TCGA-G3-A7M5-01A-11D-A33Q-10    4802    BCM GRCh38

IPP TCGA-G3-A7M5-01A-11D-A33Q-10    3652    BCM GRCh38



==> TCGA-O8-A75V-01A-11D-A32G-10.txt <==

Hugo_Symbol Tumor_Sample_Barcode    Entrez_Gene_Id  Center  NCBI_Build

TMEM51  TCGA-O8-A75V-01A-11D-A32G-10    55092   BCM GRCh38

FLG TCGA-O8-A75V-01A-11D-A32G-10    2312    BCM GRCh38

FLG TCGA-O8-A75V-01A-11D-A32G-10    2312    BCM GRCh38



To adjust a filename based on 15-char sequence of barcode value:



awk 'NR==1{ h=$0 }NR>1{ print (!a[$2]++? h ORS $0 : $0) > substr($2, 1, 15)".txt" }' file





share|improve this answer















Awk solution:



awk 'NR==1{ h=$0 }NR>1{ print (!a[$2]++? h ORS $0 : $0) > $2".txt" }' file




  • NR==1{ h=$0 } - capture the 1st line/record as header line (NR points to a record number, $0 - contains the current line)


  • NR > 1 - for all records except the first one:



    • <cond>? <operand_1> : <operand_2> - classical ternary operator


    • !a[$2]++? - check for the 1st occurrence of barcode value $2 used as a key of associative array a


    • h ORS $0 - common header line concatenated with ORS(output record separator, defaults to n) and current record $0


    • print ... > $2".txt" - print custom content or the current line(if nothing was specified) into file <barcode_value>.txt





Or a more self-explanatory version:



awk 'NR==1 {header = $0; next}

     !header_printed[$2]++ {print header > $2".txt"}

     {print > $2".txt"}' < file


Viewing results:



$ head TCGA*.txt

==> TCGA-BD-A2L6-01A-11D-A20W-10.txt <==

Hugo_Symbol Tumor_Sample_Barcode    Entrez_Gene_Id  Center  NCBI_Build

MTHFR   TCGA-BD-A2L6-01A-11D-A20W-10    4524    BCM GRCh38

SLC30A1 TCGA-BD-A2L6-01A-11D-A20W-10    7779    BCM GRCh38

USH2A   TCGA-BD-A2L6-01A-11D-A20W-10    7399    BCM GRCh38

SOS1    TCGA-BD-A2L6-01A-11D-A20W-10    6654    BCM GRCh38



==> TCGA-G3-A7M5-01A-11D-A33Q-10.txt <==

Hugo_Symbol Tumor_Sample_Barcode    Entrez_Gene_Id  Center  NCBI_Build

PRDM16  TCGA-G3-A7M5-01A-11D-A33Q-10    63976   BCM GRCh38

DNAJC11 TCGA-G3-A7M5-01A-11D-A33Q-10    55735   BCM GRCh38

HNRNPCL2    TCGA-G3-A7M5-01A-11D-A33Q-10    440563  BCM GRCh38

C1orf94 TCGA-G3-A7M5-01A-11D-A33Q-10    84970   BCM GRCh38

NFYC    TCGA-G3-A7M5-01A-11D-A33Q-10    4802    BCM GRCh38

IPP TCGA-G3-A7M5-01A-11D-A33Q-10    3652    BCM GRCh38



==> TCGA-O8-A75V-01A-11D-A32G-10.txt <==

Hugo_Symbol Tumor_Sample_Barcode    Entrez_Gene_Id  Center  NCBI_Build

TMEM51  TCGA-O8-A75V-01A-11D-A32G-10    55092   BCM GRCh38

FLG TCGA-O8-A75V-01A-11D-A32G-10    2312    BCM GRCh38

FLG TCGA-O8-A75V-01A-11D-A32G-10    2312    BCM GRCh38



To adjust a filename based on 15-char sequence of barcode value:



awk 'NR==1{ h=$0 }NR>1{ print (!a[$2]++? h ORS $0 : $0) > substr($2, 1, 15)".txt" }' file






share|improve this answer














share|improve this answer



share|improve this answer








edited Jan 31 '18 at 13:53

























answered Jan 31 '18 at 13:06









RomanPerekhrestRomanPerekhrest

22.9k12346




22.9k12346













  • Thank you very much !! Could you please explain the command?

    – user3351523
    Jan 31 '18 at 13:13













  • Please explain the command and could you also tell me how to get the output files with only 0-15 substring in the sample names like TCGA-BD-A2L6-01.txt, TCGA-G3-A7M5-01.txt and TCGA-O8-A75V-01.txt

    – user3351523
    Jan 31 '18 at 13:31






  • 1





    @user3351523, yes, see my explanation

    – RomanPerekhrest
    Jan 31 '18 at 13:47



















  • Thank you very much !! Could you please explain the command?

    – user3351523
    Jan 31 '18 at 13:13













  • Please explain the command and could you also tell me how to get the output files with only 0-15 substring in the sample names like TCGA-BD-A2L6-01.txt, TCGA-G3-A7M5-01.txt and TCGA-O8-A75V-01.txt

    – user3351523
    Jan 31 '18 at 13:31






  • 1





    @user3351523, yes, see my explanation

    – RomanPerekhrest
    Jan 31 '18 at 13:47

















Thank you very much !! Could you please explain the command?

– user3351523
Jan 31 '18 at 13:13







Thank you very much !! Could you please explain the command?

– user3351523
Jan 31 '18 at 13:13















Please explain the command and could you also tell me how to get the output files with only 0-15 substring in the sample names like TCGA-BD-A2L6-01.txt, TCGA-G3-A7M5-01.txt and TCGA-O8-A75V-01.txt

– user3351523
Jan 31 '18 at 13:31





Please explain the command and could you also tell me how to get the output files with only 0-15 substring in the sample names like TCGA-BD-A2L6-01.txt, TCGA-G3-A7M5-01.txt and TCGA-O8-A75V-01.txt

– user3351523
Jan 31 '18 at 13:31




1




1





@user3351523, yes, see my explanation

– RomanPerekhrest
Jan 31 '18 at 13:47





@user3351523, yes, see my explanation

– RomanPerekhrest
Jan 31 '18 at 13:47


















draft saved

draft discarded




















































Thanks for contributing an answer to Unix & Linux Stack Exchange!


  • Please be sure to answer the question. Provide details and share your research!

But avoid …



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f420938%2fhow-to-split-a-single-file-into-multiple-files-based-on-a-column-in-linux%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







-

Popular posts from this blog

サソリ

Image
この記事は検証可能な参考文献や出典が全く示されていないか、不十分です。 出典を追加して記事の信頼性向上にご協力ください。 （ 2019年1月 ） 「 サソリ 」のその他の用法については「サソリ (曖昧さ回避)」をご覧ください。 「 スコーピオン 」と「 スコルピオン 」はこの項目へ転送されています。その他の用法については「スコーピオン」をご覧ください。 サソリ目 生息年代: 430–0 Ma PreЄ Є O S D C P T J K Pg N シルル紀前期-現世 ダイオウサソリ（ Pandinus Imperator ） 分類 界 : 動物界 Animalia 門 : 節足動物門 Arthropoda 亜門 : 鋏角亜門 Chelicerata 綱 : クモガタ綱 Arachnida 目 : サソリ目 Scorpiones 学名 Scorpiones C. L. Koch, 1837 英名 Scorpion 上科 Pseudochactoidea キョクトウサソリ上科 Buthoidea Chaeriloidea Chactoidea Iuroidea Scorpionoidea サソリ （ サソリ類 、 蠍 、 蝎 、 scorpion ）は、鋏角亜門・クモガタ綱の サソリ目 （ Scorpiones ）に属する節足動物の総称である。体の前端に鋏型の触肢、後端に毒針を有する捕食者である。 1700種以上を含め、最古の化石記録は4億3千万年前のシルル紀まで遡る。有毒生物として広く知られるが、人間の命にかかわる毒を持つものはごく一部の種類に過ぎない。 目次 1 形態 1.1 前体 1.2 後体 1.2.1 中体 1.2.2 終体 2 雌雄 3 生態 3.1 天敵 3.2 生殖 4 分布 5 毒 6 蛍光現象 7 分類
Read more

広島県道265号伴広島線

Image
一般県道 広島県道265号伴広島線 路線延長 km 起点 広島県広島市安佐南区伴東7丁目 終点 広島県広島市中区榎町 接続する 主な道路 （記法） 広島県道38号広島豊平線 広島県道71号広島湯来線 山陽自動車道五日市IC 国道183号（ 国道261号重複） ■テンプレート（■ノート ■使い方） ■PJ道路 広島県道265号伴広島線 （ひろしまけんどう265ごう ともひろしません）は広島県広島市安佐南区から広島県広島市中区を結ぶ一般県道である。 目次 1 路線データ 2 通過する自治体 3 見どころ 4 接続道路 4.1 本線 4.2 旧道 5 別名 6 関連項目 路線データ 本線 起点：広島市安佐南区伴東7丁目・大原下橋南詰交差点（広島県道38号広島豊平線（本線・旧道）交点） 終点：広島市中区榎町・十日市交差点（国道183号（国道261号重用）交点） 総延長： 旧道 起点：広島市安佐南区伴東4丁目・新細坂交差点（広島県道38号広島豊平線交点） 終点：広島市安佐南区伴中央4丁目・寺組橋(南)交差点（広島県道265号伴広島線（本線）交点） 総延長： 通過する自治体 広島県 広島市（安佐南区、佐伯区、西区、中区） 見どころ 広島高速交通広島新交通1号線（アストラムライン） 広島広域公園 広島広域公園陸上競技場（広島ビッグアーチ） 己斐峠 己斐橋（太田川放水路） 天満橋（天満川） 接続道路 本線 広島県道38号広島豊平線（本線・旧道）（広島市安佐南区伴東7丁目・大原下橋南詰交差点〔起点〕） 広島県道71号広島湯来線（本線）（広島市安佐南区伴中央2丁目・伴交番入口交差点 - 広島市安佐南区伴中央4丁目・寺組橋(南)交差点間で重用） 広島県道71号広島湯来線（旧道）（広島県道265号伴広島線（旧道）重用）（広島市安佐南区伴中央4丁目・寺組橋(南)交差点） 広島県道71号広島湯来線（広島市安佐南区伴中央4丁目・寺組橋(南)交差点 - 広島市佐伯区五日市町石内・五月が丘(南)交差点間で重用）
Read more

Accessing regular linux commands in Huawei's Dopra Linux

Image
4 Backstory: I have a Huawei HG8245 router in my house and I want to change the default username/password. I've tried following this guide but the config file always gets reset. My ISP isn't helping either so I'm looking for other ways to accomplish this. The router has telnet access so this seems like a way to change the credentials. The first shell I gain access to looks like this: WAP> Pressing "?" gives me a list of available commands which I've pasted here, the two interesting ones are "su" and "shell". I can't seem to get su access, even after logging in as root (challenge password prompt, and then nothing happens), but shell gives me a proper linux shell. WAP>shell BusyBox v1.18.4 (2017-12-26 17:06:34 CST) built-in shell (ash) Enter 'hel
Read more