Skip to content

testfiles: consider reevaluating all files to ensure runtime requirement is met #3141

Description

@mike-hunhoff

mandiant/capa-testfiles#322 modified multiple files, some of which were committed a long time ago, which triggered the runtime check to execute. This identified that at least three of the files do not meet our current runtime requirements:

ERROR    capa.tests.data: capa ran for 308.1699028015137    check_runtimes.py:61
         seconds on                                                             
         b5f0524e69b3a3cf636c7ac366ca57bf5e3a8fdc8a9f01caf1                     
         96c611a7918a87.elf_, please provide a different                        
         sample so we can test more quickly  

ERROR    capa.tests.data: capa ran for 374.43834686279297   check_runtimes.py:61
         seconds on 92d8ea10ea30e8b534334a1c9857a455.exe_,                      
         please provide a different sample so we can test                       
         more quickly     

ERROR    capa.tests.data: capa ran for 275.35956621170044   check_runtimes.py:61
         seconds on                                                             
         559efe9f3d4864910d5c5edbaadd3972c6dfc3ba887c48aa67                     
         28b0093898563f.dll_, please provide a different                        
         sample so we can test more quickly   

These three files alone account for 15 minutes of runtime. The runtime check only executes for the files modified/added in a given pull request, so there are likely other files that have flown under the radar. IIRC this is only an issue when a thorough test is manually triggered one or more times before a new release. We should reevaluate all test files and replace those that no longer meet our runtime requirements if thorough test runtime becomes a concern.

Metadata

Metadata

Assignees

No one assigned

    Labels

    CIContinuous Integration configurationenhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions