Post

Replies

Boosts

Views

Activity

Memory mapped file: "Cannot allocate memory"
Hello,I am having an hard time figuring out how memory mapped files works under iOS. Suppose that I want to read a big file, let's say 4GB. If I use memory mapped files in READ mode I should be able to get a valid pointer to a file and benefit from the page fault mechanism of virtual memory but apparently iOS can map only up to 2.5GB of data. I run the following test on an iPad Pro A1673 that has 2GB of ram and got as result:2018-04-06 17:20:12.660888+0200 TestMemory[414:316466] Data address: 0x1022240002018-04-06 17:20:12.662355+0200 TestMemory[414:316466] Data address: 0x1122240002018-04-06 17:20:12.663801+0200 TestMemory[414:316466] Data address: 0x12b9000002018-04-06 17:20:12.665235+0200 TestMemory[414:316466] Data address: 0x13b9000002018-04-06 17:20:12.666756+0200 TestMemory[414:316466] Data address: 0x14b9000002018-04-06 17:20:12.668202+0200 TestMemory[414:316466] Data address: 0x15b9000002018-04-06 17:20:12.669597+0200 TestMemory[414:316466] Data address: 0x16ff240002018-04-06 17:20:12.739549+0200 TestMemory[414:316466] Data address: 0x1e00000002018-04-06 17:20:12.747604+0200 TestMemory[414:316466] Data address: 0x1f00000002018-04-06 17:20:12.749130+0200 TestMemory[414:316466] Data address: 0x2000000002018-04-06 17:20:12.764753+0200 TestMemory[414:316466] NSData failed: Error Domain=NSCocoaErrorDomain Code=256 "The file “02808580-5D42-43BC-B5AA-628E3682A546” couldn’t be opened." UserInfo={NSFilePath=/var/mobile/Containers/Data/Application/5207275C-5525-47CA-AC4B-2AA07E05C1E9/Documents/02808580-5D42-43BC-B5AA-628E3682A546, NSUnderlyingError=0x1c0455270 {Error Domain=NSPOSIXErrorDomain Code=12 "Cannot allocate memory"}}2018-04-06 17:20:12.764801+0200 TestMemory[414:316466] Mapped 2684354560- (NSString*)createFileOfSize:(unsigned long long)size { NSString* uuid = [[NSUUID UUID] UUIDString]; NSString* documentFolderPath = [[[NSFileManager defaultManager] URLsForDirectory:NSDocumentDirectory inDomains:NSUserDomainMask] lastObject].path; NSString* filePath = [documentFolderPath stringByAppendingPathComponent:uuid]; NSFileHandle *fileHandle = [NSFileHandle fileHandleForWritingAtPath:filePath]; if (fileHandle == nil) { [[NSFileManager defaultManager] createFileAtPath:filePath contents:nil attributes:nil]; fileHandle = [NSFileHandle fileHandleForWritingAtPath:filePath]; } [fileHandle truncateFileAtOffset:size]; [fileHandle closeFile]; return filePath; } - (void)test { const unsigned long long MB = 1 << 20; const unsigned long long GB = 1 << 30; unsigned long long space = 4 * GB; unsigned long long size = 64 * MB; unsigned long long fileCount = space / size; NSMutableArray* mapped = [NSMutableArray array]; int i = 0; for (i = 0; i < fileCount; i++) { NSString* filePath = [self createFileOfSize:size]; NSError* error = nil; NSData* data = [NSData dataWithContentsOfFile:filePath options:NSDataReadingMappedAlways error:&error]; if (error) { NSLog(@"NSData failed: %@", error); break; } else { const void* bytes = [data bytes]; NSLog(@"Data address: %p", bytes); [mapped addObject:data]; } } NSLog(@"Mapped %llu", i * size); }Any idea of why we have this limitation?Thanks!Libe
10
0
8.1k
Sep ’22
Memory mapped file: "Cannot allocate memory"
Hello,I am having an hard time figuring out how memory mapped files works under iOS. Suppose that I want to read a big file, let's say 4GB. If I use memory mapped files in READ mode I should be able to get a valid pointer to a file and benefit from the page fault mechanism of virtual memory but apparently iOS can map only up to 2.5GB of data. I run the following test on an iPad Pro A1673 that has 2GB of ram and got as result:2018-04-06 17:20:12.660888+0200 TestMemory[414:316466] Data address: 0x1022240002018-04-06 17:20:12.662355+0200 TestMemory[414:316466] Data address: 0x1122240002018-04-06 17:20:12.663801+0200 TestMemory[414:316466] Data address: 0x12b9000002018-04-06 17:20:12.665235+0200 TestMemory[414:316466] Data address: 0x13b9000002018-04-06 17:20:12.666756+0200 TestMemory[414:316466] Data address: 0x14b9000002018-04-06 17:20:12.668202+0200 TestMemory[414:316466] Data address: 0x15b9000002018-04-06 17:20:12.669597+0200 TestMemory[414:316466] Data address: 0x16ff240002018-04-06 17:20:12.739549+0200 TestMemory[414:316466] Data address: 0x1e00000002018-04-06 17:20:12.747604+0200 TestMemory[414:316466] Data address: 0x1f00000002018-04-06 17:20:12.749130+0200 TestMemory[414:316466] Data address: 0x2000000002018-04-06 17:20:12.764753+0200 TestMemory[414:316466] NSData failed: Error Domain=NSCocoaErrorDomain Code=256 "The file “02808580-5D42-43BC-B5AA-628E3682A546” couldn’t be opened." UserInfo={NSFilePath=/var/mobile/Containers/Data/Application/5207275C-5525-47CA-AC4B-2AA07E05C1E9/Documents/02808580-5D42-43BC-B5AA-628E3682A546, NSUnderlyingError=0x1c0455270 {Error Domain=NSPOSIXErrorDomain Code=12 "Cannot allocate memory"}}2018-04-06 17:20:12.764801+0200 TestMemory[414:316466] Mapped 2684354560- (NSString*)createFileOfSize:(unsigned long long)size { NSString* uuid = [[NSUUID UUID] UUIDString]; NSString* documentFolderPath = [[[NSFileManager defaultManager] URLsForDirectory:NSDocumentDirectory inDomains:NSUserDomainMask] lastObject].path; NSString* filePath = [documentFolderPath stringByAppendingPathComponent:uuid]; NSFileHandle *fileHandle = [NSFileHandle fileHandleForWritingAtPath:filePath]; if (fileHandle == nil) { [[NSFileManager defaultManager] createFileAtPath:filePath contents:nil attributes:nil]; fileHandle = [NSFileHandle fileHandleForWritingAtPath:filePath]; } [fileHandle truncateFileAtOffset:size]; [fileHandle closeFile]; return filePath; } - (void)test { const unsigned long long MB = 1 << 20; const unsigned long long GB = 1 << 30; unsigned long long space = 4 * GB; unsigned long long size = 64 * MB; unsigned long long fileCount = space / size; NSMutableArray* mapped = [NSMutableArray array]; int i = 0; for (i = 0; i < fileCount; i++) { NSString* filePath = [self createFileOfSize:size]; NSError* error = nil; NSData* data = [NSData dataWithContentsOfFile:filePath options:NSDataReadingMappedAlways error:&error]; if (error) { NSLog(@"NSData failed: %@", error); break; } else { const void* bytes = [data bytes]; NSLog(@"Data address: %p", bytes); [mapped addObject:data]; } } NSLog(@"Mapped %llu", i * size); }Any idea of why we have this limitation?Thanks!Libe
Replies
10
Boosts
0
Views
8.1k
Activity
Sep ’22