Memory allocation with GR8NET and Fusion-C

بواسطة klimchuk

Supporter (13)

صورة klimchuk

29-04-2022, 05:33

I'm writing application that connects to the server using HTTP protocol. In order to do that I've used great example at https://github.com/ericb59/Fusion-C-v1.2/blob/master/Working... and everything works fine if size of executable com file stays below 22Kb. If com file becomes larger a=tcpip_tcp_open(&tcp_conn_parms,&conn_number); call starts randomly return 2 or 4 or anything else. And that's the only thing that doesn't work in the whole application. Is UNAPI implementation for GR8NET takes particular space in memory?
Or may be it's just MSX-DOS limitation?

Login أوregister لوضع تعليقاتك

بواسطة aoineko

Paragon (1137)

صورة aoineko

29-04-2022, 08:57

According to the MSX Technical Handbook (https://konamiman.github.io/MSX2-Technical-Handbook/md/Chapt...) the area a program can use under MSX-DOS is between 0100h and the value at (0006h). I suppose it's more than 22 KB but I'm not sure (I suppose it also depend on the number of drive installed).

بواسطة erikmaas

Expert (70)

صورة erikmaas

29-04-2022, 10:09

I think you should try to keep the data outside the range of 0x4000-0x7fff when doing UNAPI calls.

بواسطة ducasp

Paladin (712)

صورة ducasp

29-04-2022, 16:36

klimchuk wrote:

I'm writing application that connects to the server using HTTP protocol. In order to do that I've used great example at https://github.com/ericb59/Fusion-C-v1.2/blob/master/Working... and everything works fine if size of executable com file stays below 22Kb. If com file becomes larger a=tcpip_tcp_open(&tcp_conn_parms,&conn_number); call starts randomly return 2 or 4 or anything else. And that's the only thing that doesn't work in the whole application. Is UNAPI implementation for GR8NET takes particular space in memory?
Or may be it's just MSX-DOS limitation?

The way UNAPI works, the driver is sitting on its own memory block that is not the one DOS/Applications are using, it will either be a memory from the cartridge on the slot or a memory mapper segment that is reserved and exclusive to the driver. When you call an UNAPI function, that will be paged into 4000-7FFF, so, you must make sure that any data you pass to it is not within that range (as the UNAPI itself will be in that block, so, your data won't be visible to it). There is a third option, that the driver resides in C000 to FFFF area, that means, using TPA memory, never saw anyone doing it and hopefully that is never used, but if someone uses it (absolutely sure that GR8NET doesn't do it) it will decrease the ammount of memory you can use to your application.

https://github.com/Konamiman/MSX-UNAPI-specification/blob/ma...

So, perhaps, when your application is getting larger, some of the buffers you are using to pass/receive data are being moved within 4000-7FFF memory range and causing troubles... :P

By the way, if you are using http protocol, perhaps you might want to look at this:

https://github.com/ducasp/MSX-Development/tree/master/UNAPI/...

In there, I've transformed HGET in a "library", so, basically, that is the tried and tested http code reference for MSX by Konamiman with a few nice tweaks, like allowing a callback function to receive data blocks, allowing a callback function to receive updated % progress in 4% steps, also another callback to update received file size (if you just ask to save the contents to a file) and allow keep alive connections (which is quite handy if you are transferring multiple files at once, so you do not need to open a connection for each file)

Hopefully that helps you. ;)

بواسطة klimchuk

Supporter (13)

صورة klimchuk

29-04-2022, 20:39

Thank you for helpful answers!
Unfortunately avoiding 0x4000-0x7fff means I have less than 16Kb of memory for com file. I could probably fit my com into 26Kb but there is no way to cut 10Kb out of it without sacrificing half of functionality. Also SDCC is putting global variables to the end of com file and they definitely go beyond 0x4000 in memory breaking UNAPI.

Can Nextor move loaded com file to extended RAM of the GR8NET and run it from that location?

بواسطة ducasp

Paladin (712)

صورة ducasp

29-04-2022, 21:17

klimchuk wrote:

Thank you for helpful answers!
Unfortunately avoiding 0x4000-0x7fff means I have less than 16Kb of memory for com file. I could probably fit my com into 26Kb but there is no way to cut 10Kb out of it without sacrificing half of functionality. Also SDCC is putting global variables to the end of com file and they definitely go beyond 0x4000 in memory breaking UNAPI.

Can Nextor move loaded com file to extended RAM of the GR8NET and run it from that location?

It doesn't mean that... Note that the only restriction is that the memory area you pass to UNAPI functions can't be on that position, your code can use it fully, as once UNAPI function returns it will switch back the memory to your program segment.

There are several ways to accomplish that, I'm lazy, so I use the easier one:

In .h file:

//Where we will allocate memory for hget and other processes
#define HI_MEMBLOCK_START 0xC000

On the c file:

// Allocate memory for HGET on page 2 so it won't conflict with memory page swapping
if (hgetinit(HI_MEMBLOCK_START) != ERR_TCPIPUNAPI_OK)

Of course, that limits my program to about 48KB, but if you check the map after building your program, you can adjust that.

There are more elegant ways like this one:

__at 0x8000 unsigned char ucRcvDataMemory[]; //area to hold data sent to UNAPI, need to be in the 3rd 16K block

Not sure why I don't use it on every program I do, but that should also guarantee that variable to be allocated where you want, perhaps there was a sdcc bug somewhere with it, but it works most of the times just fine as well Smile

So, just remember, any memory address / variable you pass as parameter for those UNAPI calls MUST NOT BE ON PAGE 1 (considering 0 0000 to 3FFF and 1 4000 to 7FFF), you can have other variables and program information there AS LONG AS it is not passed as a parameter for UNAPI calls. Wink

بواسطة erikmaas

Expert (70)

صورة erikmaas

29-04-2022, 21:33

I also had this issue for data for HTTP requests. In my case I was a bit lazy and made a glue layer that transfers the data via a small stack buffer and repeats this until the buffer is depleted. This makes sure it always works, although not the fasted. At least you can continue with your application.
Since you have it failing on the connect, then using a small stack buffer as an intermediate step should cause not so much harm performance wise.

(I see I am slow at typing a response, and we are all lazy people Eek! .... I did not find the solution that ducasp suggested here yet, I am also definitely going to use that fixed address definition)

بواسطة klimchuk

Supporter (13)

صورة klimchuk

30-04-2022, 00:21

@ducasp

Bingo!

__at 0x8000 struct DD d[10];

did fix the issue with UNAPI and now HTTP communication is rock solid!
I still see some corruption inside d[10] so perhaps 0x8000 is not an ideal location but at least it solved my biggest obstacle to move forward.

Thank you very much!

بواسطة ducasp

Paladin (712)

صورة ducasp

30-04-2022, 00:59

klimchuk wrote:

@ducasp

Bingo!

__at 0x8000 struct DD d[10];

did fix the issue with UNAPI and now HTTP communication is rock solid!
I still see some corruption inside d[10] so perhaps 0x8000 is not an ideal location but at least it solved my biggest obstacle to move forward.

Thank you very much!

Quite possibly sdcc is not being smart enough and that structure is being placed over some function or other variable and thus why it is getting overwritten. Check after compiling if there is something else occupying a similar space. My advise would be, look at the map, and place it way far from the last position occupied... There is one problem with this approach, your executable probably will as large as the address you choose, so, you can move it one KB or two after the end of your program, so it will not inflate the file size too much... If you just define the memory like I showed before, at 0xC000 and then cast your structure pointer to it and use*d when parsing the structure (since you will declare it as a pointer), file size won't inflate and your variable will be far, far away from code, sdcc area for variables, etc... Not beauty, but easy to have it in a place you know will work and won't cause any nuisance. Tongue

بواسطة ducasp

Paladin (712)

صورة ducasp

30-04-2022, 00:59

And there is always a possibility you have a bug in your code, of course LOL!

بواسطة Eugeny_Brychkov

Paragon (1232)

صورة Eugeny_Brychkov

21-06-2022, 17:35

Hope issues were solved. If there're still ones please contact me.