通过postman执行stream load parquet文件导入失败

Viewed 19

image.png

{
    "TxnId": 3991927,
    "Label": "5386319b-3730-4458-8ed4-3852a99e716a",
    "Comment": "",
    "TwoPhaseCommit": "false",
    "Status": "Fail",
    "Message": "[CORRUPTION]Invalid magic number in parquet file, bytes read: 131072, file size: 20828303, path: /mnt/doris/be_storage/mini_download/bdl_el_company/__shard_743//court_lesscredit.20241220165522.125021, read magic: 。å\n\n\t0#  doris::vectorized::parse_thrift_footer(std::shared_ptr<doris::io::FileReader>, doris::vectorized::FileMetaData**, unsigned long*, doris::io::IOContext*) at /home/zcp/repo_center/doris_enterprise/doris/be/src/common/status.h:380\n\t1#  doris::FileMetaCache::get_parquet_footer(std::shared_ptr<doris::io::FileReader>, doris::io::IOContext*, long, unsigned long*, doris::ObjLRUCache::CacheHandle*) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:701\n\t2#  doris::vectorized::ParquetReader::_open_file() at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:701\n\t3#  doris::vectorized::ParquetReader::open() at /home/zcp/repo_center/doris_enterprise/doris/be/src/common/status.h:446\n\t4#  doris::vectorized::VFileScanner::_get_next_reader() at /home/zcp/repo_center/doris_enterprise/doris/be/src/common/status.h:446\n\t5#  doris::vectorized::VFileScanner::_get_block_impl(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_enterprise/doris/be/src/common/status.h:446\n\t6#  doris::vectorized::VScanner::get_block(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/scan/vscanner.cpp:0\n\t7#  doris::vectorized::ScannerScheduler::_scanner_scan(doris::vectorized::ScannerScheduler*, doris::vectorized::ScannerContext*, std::shared_ptr<doris::vectorized::VScanner>) at /home/zcp/repo_center/doris_enterprise/doris/be/src/common/status.h:357\n\t8#  std::_Function_handler<void (), doris::vectorized::ScannerScheduler::_schedule_scanners(doris::vectorized::ScannerContext*)::$_1::operator()() const::{lambda()#4}>::_M_invoke(std::_Any_data const&) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:701\n\t9#  doris::WorkThreadPool<true>::work_thread(int) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:646\n\t10# execute_native_thread_routine at /data/gcc-11.1.0/build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/unique_ptr.h:85\n\t11# start_thread\n\t12# clone\n",
    "NumberTotalRows": 0,
    "NumberLoadedRows": 0,
    "NumberFilteredRows": 0,
    "NumberUnselectedRows": 0,
    "LoadBytes": 20828303,
    "LoadTimeMs": 7588,
    "BeginTxnTimeMs": 0,
    "StreamLoadPutTimeMs": 2,
    "ReadDataTimeMs": 76,
    "WriteDataTimeMs": 550,
    "CommitAndPublishTimeMs": 0
}
2 Answers

是否在发送的header里面加上了格式:

format:Parquet