PDA

View Full Version : Possibly malfunctioning h7s



chrcoluk
14-08-21, 18:04
Some of you may remember before I reported I had to auto reboot the box every 3 days to keep it working.

Well I have been observing the past few months that on iptv I would get a picture freeze every 2-3 seconds, the obvious thought is a network problem right?

Well if I watch 'top' in the shell every 2-3 seconds the 'si' usage goes to 100%. The software interrupts counter.

I have just clean flashed the latest vix 5.4 image and its still happening.

Given how highly rated these units are I am starting to think mine is defective, but I am hoping someone may know of a workaround to the problem.


# top
top - 18:05:38 up 10 min, 1 user, load average: 0.06, 0.04, 0.00
Tasks: 117 total, 1 running, 116 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.0 us, 0.0 sy, 0.0 ni, 0.0 id, 0.0 wa, 0.0 hi,100.0 si, 0.0 st
MiB Mem : 1004.0 total, 174.9 free, 729.9 used, 99.2 buff/cache
MiB Swap: 256.0 total, 256.0 free, 0.0 used. 241.3 avail Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2051 root 20 0 145292 73808 15600 S 0.7 7.2 0:12.22 enigma2
2193 root 20 0 6288 2276 1940 R 0.7 0.2 0:01.39 top
7 root 20 0 0 0 0 S 0.3 0.0 0:00.07 rcu_sched
1452 root 20 0 0 0 0 S 0.3 0.0 0:01.58 nxsched
1 root 20 0 1604 328 284 S 0.0 0.0 0:01.39 init
2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd
4 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/0:0H
6 root 20 0 0 0 0 S 0.0 0.0 0:00.08 ksoftirqd/0
8 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcu_bh
9 root rt 0 0 0 0 S 0.0 0.0 0:00.00 migration/0
10 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 lru-add-drain
11 root rt 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/0
12 root 20 0 0 0 0 S 0.0 0.0 0:00.00 cpuhp/0
13 root 20 0 0 0 0 S 0.0 0.0 0:00.00 cpuhp/1
14 root rt 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/1
15 root rt 0 0 0 0 S 0.0 0.0 0:00.00 migration/1
16 root 20 0 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/1

twol
14-08-21, 18:18
Which plugins have you installed???

chrcoluk
14-08-21, 18:20
Can a plugin cause software interrupts?

List of plugins is.

crossepg
iptv bouquet maker
open webif

chrcoluk
15-08-21, 01:57
On the new version of vix I have left the auto reboot disabled, and will see what happens on that particular issue.

The iptv is not confirmed working or not as I havent reinstalled the plugins for it.

I did find a thread elsewhere with people discussing hanging h7s boxes a few of them, and they all had faulty receiver's.

chrcoluk
15-08-21, 21:27
A little update, I have identified over committing of memory, the unit even when firstly booted up has under 200 meg of free ram, the committed memory isnt on temporary things like cache is its fully committed to things that cannot be removed from memory if needed.

Due to the memory constraints I looked into ways to ease the problem. With a swap file installed and default vm.swappiness, it routinely will use the swap file setting the swapinness to as low as 5 is not utilising it, removing the swapfile causes the box to crash with OOM.

Part of the problem has also been identified the use of /var/volatile/tmp using a ram disk, its default sized to 500meg, this is where I made some progress, the disk is backed by swap, so wont error out when is not enough memory providing there is a swap device present, the default size of 500meg when the system only has circa 200 meg of memory to allocate is obviously a disaster waiting to happen, interestingly when I capped the ram disk to 100meg enigma crashed over night and went into a boot loop so I had proof enigma was doing something that required a lot of tmp storage (in ram), and it was epg updates.

I have since made a tmp dir on the hdd and bind mounted it over /var/volatile/tmp, existing /tmp is symlinked to /var/volatile/tmp, I thought about changing that symlink but I expect that was too risky with system updates undoing that change.

There has been some clear overconfidence in the capabilities of this unit in terms of ram, I feel selling a device with only 1 gig of ram in 2021 is a bit questionable, the manufacturers of this device probably should rethink that one. The amount of stories on the net of this unit been unstable is starting to make some sense, although at least some of those stories confirmed to be faulty receiver chips.

So will now see if this can stay stable overnight for a decent period of time, hopefully I wont report back for a week or two and only then to confirm no more problems. :) A usb stick is been ordered as using the hdd for tmp has a little risk if it fills up with recordings.

In addition cat /proc/interrupts shows a bunch of strangely named processes with varying length of 'b' in their name. Seems lots of oddly proprietary stuff running that is hard to debug including the earlier identified '999999999999999' process.

I am not sure if this is a clean install of vix as I did the upgrade via image manager and chose not to restore the backed up config, I may still do an install from the recovery image if instability prevails. Also htop oddly doesnt show the 100% usage that normal top shows under 'si' usage.

ccs
15-08-21, 21:45
Where do you store epg.dat? If it's in flash, then move it to hdd or usb.

ronand
16-08-21, 09:19
A little update, I have identified over committing of memory, the unit even when firstly booted up has under 200 meg of free ram, the committed memory isnt on temporary things like cache is its fully committed to things that cannot be removed from memory if needed.

Due to the memory constraints I looked into ways to ease the problem. With a swap file installed and default vm.swappiness, it routinely will use the swap file setting the swapinness to as low as 5 is not utilising it, removing the swapfile causes the box to crash with OOM.

Part of the problem has also been identified the use of /var/volatile/tmp using a ram disk, its default sized to 500meg, this is where I made some progress, the disk is backed by swap, so wont error out when is not enough memory providing there is a swap device present, the default size of 500meg when the system only has circa 200 meg of memory to allocate is obviously a disaster waiting to happen, interestingly when I capped the ram disk to 100meg enigma crashed over night and went into a boot loop so I had proof enigma was doing something that required a lot of tmp storage (in ram), and it was epg updates.

I have since made a tmp dir on the hdd and bind mounted it over /var/volatile/tmp, existing /tmp is symlinked to /var/volatile/tmp, I thought about changing that symlink but I expect that was too risky with system updates undoing that change.

There has been some clear overconfidence in the capabilities of this unit in terms of ram, I feel selling a device with only 1 gig of ram in 2021 is a bit questionable, the manufacturers of this device probably should rethink that one. The amount of stories on the net of this unit been unstable is starting to make some sense, although at least some of those stories confirmed to be faulty receiver chips.

So will now see if this can stay stable overnight for a decent period of time, hopefully I wont report back for a week or two and only then to confirm no more problems. :) A usb stick is been ordered as using the hdd for tmp has a little risk if it fills up with recordings.

In addition cat /proc/interrupts shows a bunch of strangely named processes with varying length of 'b' in their name. Seems lots of oddly proprietary stuff running that is hard to debug including the earlier identified '999999999999999' process.

I am not sure if this is a clean install of vix as I did the upgrade via image manager and chose not to restore the backed up config, I may still do an install from the recovery image if instability prevails. Also htop oddly doesnt show the 100% usage that normal top shows under 'si' usage.

I don't see a flood of stories on the net about the h7. Mine has never run out of memory with the default 256MB swap.

chrcoluk
16-08-21, 17:27
Where do you store epg.dat? If it's in flash, then move it to hdd or usb.

on the hdd, that has no bearing on the memory consumption, during the process of creating the epg there is temporary files written to /tmp and /tmp is hosted on a ramdisk.

The vix crash logs were very useful in diagnosing this.

The initial crashes reported /tmp had ran out of space to store epg files (after I reduced it to 100meg thinking vix wouldnt need so much scratch space). The decision to reduce it was a bad one but did serve to help me debug what is going on, the ram disk only consumes ram for used space. It is backed by swap if there isnt enough ram.

Also thanks guys for confirming swap is on by default, as due to me been unsure if this is an actual clean vix install, I wasnt sure if I added the swap or it was default. It is 256meg, so I am assuming now its a default enabled swap.

After I did the bind mount moving tmp to my hdd, the crashes stopped, thus far since I made those changes the box has been fine and responsive, but I need more than 1-2 days to consider if its working ok, if it makes a week, then this is great progress as before I had to automate reboots every 2-3 days.

There is currently nearly 600 gig in my trash of recordings which will be deleted probably in next day or so and currently 40 gig of free space, so hdd wont be full for a while now. By then I will have the usb taking over /tmp duties.

ccs
16-08-21, 17:36
on the hdd, that has no bearing on the memory consumption, during the process of creating the epg there is temporary files written to /tmp and /tmp is hosted on a ramdisk.

If it's on the hdd then there's no problem, but the default is to save an epg copy in a file called epg.dat in internal flash, which can cause problems.

BrokenUnusableAccount
16-08-21, 20:46
I don't see a flood of stories on the net about the h7. Mine has never run out of memory with the default 256MB swap.

I'd never really investigated how much memory was free, or the swapfile.

It seems that by default it uses /dev/mmcblk0p7 as a 256MB swap partition.

Using emmc flash as swap can't really be a good idea, if you ever run something big that makes it really thrash it'll wear out the emmc.

However I can't work out how to permanantly stop it using /dev/mmcblk0p7 as swap.


swapoff /dev/mmcblk0p7 stops it right now, but next time it boots up there it is again 256MB swap using /dev/mmcblk0p7 !

I'd feel a lot happier using a swapfile on my hard drive.

chrcoluk
17-08-21, 00:32
Hmm I honestly dont know if I tinkered with the swap in the past but my swap is on the hdd currently, I dont think its a good idea to use the flash as swap for the same reasons I was advised to make sure the epg is not on my flash.

root@zgemmah7:/tmp# cat /proc/swaps
Filename Type Size Used Priority
/media/hdd/swapfile file 262140 320 -1

in /etc/fstab (where it is set to apply at boot), there needs to be a swapfile created first.

/media/hdd/swapfile swap swap defaults 0 0

BrokenUnusableAccount
17-08-21, 01:35
Hmm I honestly dont know if I tinkered with the swap in the past but my swap is on the hdd currently, I dont think its a good idea to use the flash as swap for the same reasons I was advised to make sure the epg is not on my flash.

root@zgemmah7:/tmp# cat /proc/swaps
Filename Type Size Used Priority
/media/hdd/swapfile file 262140 320 -1

in /etc/fstab (where it is set to apply at boot), there needs to be a swapfile created first.

/media/hdd/swapfile swap swap defaults 0 0

My H7 just completely refuses to do that.
It dosn't seem to care what's in /etc/fstab
I can't get anything except:

root@Zgemmah7:~# cat /proc/swaps
Filename Type Size Used Priority
/dev/mmcblk0p7 partition 262140 0 -1
root@Zgemmah7:~#
or no swap at all.
:(

Oh well. At least mine seems to be almost never using the swap.

chrcoluk
17-08-21, 03:06
Yeah at least you not using the swap, I got the swappiness set down to 5, and its still using 320k albeit a tiny amount. Currently only 62 meg of free ram with another 62 meg allocated to caching, its been creeping down so still curious if the box makes it a week.

Interestingly as well not all of the combined free+caching is available either. Just under 100M available. From around 200M on a fresh boot.

# free
total used free shared buff/cache available
Mem: 1028056 897756 64360 76 65940 99212
Swap: 262140 320 261820

ronand
17-08-21, 11:23
On any linux box I've had (and I've had a few) the system seems to allocate most of the available RAM no matter how much there actually is. So I wouldn't read too much into that. If you are getting OOM then there must a plugin that you are using that it doing it - there is definitely enough resources for use as a satellite receiver.

chrcoluk
17-08-21, 17:20
On any linux box I've had (and I've had a few) the system seems to allocate most of the available RAM no matter how much there actually is. So I wouldn't read too much into that. If you are getting OOM then there must a plugin that you are using that it doing it - there is definitely enough resources for use as a satellite receiver.

I agree though that as long as the box works, which is ultimately what matters then its ok, which is why I am seeing if it will work for the entire week.

How do I diagnose memory usage for plugins? right now in the UI there is only 3 basic plugins there. enigma itself is only using 110meg of ram. A lot of the memory usage seems tied up in proprietary kernel drivers.

ronand
17-08-21, 17:27
If its a faulty/buggy plugin causing the problem how is it a fault of the box? Admittedly I don't use a lot of extra plugins as it is mainly for DVB-S/T viewing and some media playback. I wouldn't waste my time with heavily compressed IPTV video and poor audio and that is where most of the crap plugins are used. There have been no complaints on this forum about low memory on this box despite your claim of being a flood of it on the web.

chrcoluk
17-08-21, 17:38
It wasnt hard to google for it, I apologise if you are unable to find the reports, you do seem to be very defensive, instead of helping to identify things, you in a "there is no problem, it is you" mode.

I am not the one blaming it on a plugin either, there has been assumptions it is a plugin but without any kind of diagnosis to lead to that.

Just to 100% and knock this theory dead, the box had the freeze problems when no IPTV was used on the box. There is no IPTV used on this box right now either, I am testing on freesat tv.

As to whether the ram is enough, well yeah we could all only use these box in pure stock configurations, but then the question why not just use a standard sky tv box? You opening a huge cvan of worms with these defensive statements.

Now if you want to help and contribute to the discussion at hand then I appreciate your help, but if you think its down to some poorly coded plugin that isnt written to work in a tiny amount of available memory then please post the substance of that from the information I have provided, thank you.

I am open to all outcomes here, it could be a rogue plugin with a memory leak (I have said as such), it could be faulty hardware, it could be bad software configuration, it could be bad drivers. This is the basis of testing and diagnosing things. I dont come to assumptions because those assumptions are convenient.

ronand
17-08-21, 17:44
I feel selling a device with only 1 gig of ram in 2021 is a bit questionable, the manufacturers of this device probably should rethink that one.

If you wanted/needed more RAM you should have shelled out for it in the first place. Its a budget/low end 4K receiver. Most of the boxes around the £100 mark have 1GB RAM and some have less. The £200 and upwards boxes have 2/3/4 GBs - there obviously has to be some reason for the price difference. You wont get a Merc for the price of a Ford Cortina I'm afraid.

ronand
17-08-21, 17:49
Its hard to know what the problem is when you haven't provided a single log. Indeed the first test is to run the receiver in a very basic state on FTA TV - no additional plugins or softcams. Disconnect the hard drive and remove any USB device. If it is still freezing then it would more likely be a hardware problem rather than a memory problem.

chrcoluk
17-08-21, 17:51
If you wanted/needed more RAM you should have shelled out for it in the first place. Its a budget/low end 4K receiver. Most of the boxes around the £100 mark have 1GB RAM and some have less. The £200 and upwards boxes have 2/3/4 GBs - there obviously has to be some reason for the price difference. You wont get a Merc for the price of a Ford Cortina I'm afraid.

I accept that view, I perhaps misjudged that the ram would be enough, and if I do diagnose that I need more ram rather than it been a rogue plugin or bad configuration I may well buy a box with more ram.

If I understand right plugins are enigma plugins so there usage would fall under engima? or am I incorrect on that one.

chrcoluk
17-08-21, 17:52
Its hard to know what the problem is when you haven't provided a single log. Indeed the first test is to run the receiver in a very basic state on FTA TV - no additional plugins or softcams. Disconnect the hard drive and remove any USB device. If it is still freezing then it would more likely be a hardware problem rather than a memory problem.

I am open to doing that when this week is up. I want to finish this week first, but then I can remove the hdd and any usb sticks, only things connected, hdmi, power and sat feed cables. Am I ok to delete all plugins in the plugin screen? So effectively running with none at all.

There has been no softcam on here for a long time unless one is lingering somewhere.

ronand
17-08-21, 17:55
Don't start deleting stuff. It may be a problem with your current setup so we need to eliminate that. Install a fresh image in one of the other multiboot slots leaving your current setup intact and just set it up for FTA TV from scratch - dont restore any settings.

chrcoluk
17-08-21, 18:08
Whats the best way to do that, if I use the usb boot image, will that pick a unused slot or let me pick, or should I be using the image manager like I did before?

ronand
17-08-21, 18:26
Use the image manager to download the latest 5.4 image and flash it - it will ask you which slot you want to use.

The purpose of all this is to see if the box is stable (ie the hardware is ok). If it runs fine for a few days then the box should be fine and you can start adding stuff back in stages to see what actually causes your problem.

lincsat
20-06-22, 17:27
I've joined this thread as I'm seeing a similar issue, the RAM is almost full and on boot the box wants to only use swap on /dev/mmcblk0p7. I do have several apps and plugins loaded and I understand that I should really have a box with more RAM but am trying to make the best out of what I have.

I've generated swapfiles on both the USB and HDD of varying sizes from 128Mbit to 1Gbit and tried various entries in fstab. On boot, the system generates a new entry in fstab for a swap on /dev/mmcblk0p7 and uses that by default.

Is there a way of stopping the system generating a new entry for the swap in fstab or giving the /media/usb/swapfile a higher priority and is it worth increasing the swappiness? Be gentle, I'm a Linux novice

twol
20-06-22, 19:00
I've joined this thread as I'm seeing a similar issue, the RAM is almost full and on boot the box wants to only use swap on /dev/mmcblk0p7. I do have several apps and plugins loaded and I understand that I should really have a box with more RAM but am trying to make the best out of what I have.

I've generated swapfiles on both the USB and HDD of varying sizes from 128Mbit to 1Gbit and tried various entries in fstab. On boot, the system generates a new entry in fstab for a swap on /dev/mmcblk0p7 and uses that by default.

Is there a way of stopping the system generating a new entry for the swap in fstab or giving the /media/usb/swapfile a higher priority and is it worth increasing the swappiness? Be gentle, I'm a Linux novice
What does swapmanager show in vix plugin…… if they are named swap###### should find them

lincsat
20-06-22, 19:29
It shows manufacturer defined and won't let me change it

64014

twol
21-06-22, 08:01
It shows manufacturer defined and won't let me change it

64014

what does it show in debug log when the ViX plugins are initialised - should list all found.

twol
21-06-22, 08:01
deleted ,,,,,,,,,,,,,,,,,

twol
21-06-22, 11:56
@Lincsat ... can you plugin the attachment ( /usr/lib/enigma2/python/Plugins/SystemPlugins/ViX ) and attach debug log --- might also give you some additional options!! BUT !! do not try deleting mmc swap!
It could be mmc swap is last in list and that's why its picked.... so need to see debug output.

lincsat
21-06-22, 12:29
I had found how to prioritise a swap and implemented that by changing the fstab entry to

/media/usb/swapfile swap swap pri=0 0 0

It is now using the USB swap at boot but still creating the MMC swap and adding that to the swap pool. It's too early to tell if the box is performing any better.


root@zgemmah7:~# cat /proc/swaps
Filename Type Size Used Priority
/dev/mmcblk0p7 partition 262140 0 -1
/media/usb/swapfile file 1048572 28 0
root@zgemmah7:~# free
total used free shared buff/cache available
Mem: 1027736 915388 33492 204 78856 79556
Swap: 1310712 28 1310684
root@zgemmah7:~#

using new swapmanager, I have attached the log. The GUI is still showing the same as previously

twol
21-06-22, 15:58
I had found how to prioritise a swap and implemented that by changing the fstab entry to

/media/usb/swapfile swap swap pri=0 0 0

It is now using the USB swap at boot but still creating the MMC swap and adding that to the swap pool. It's too early to tell if the box is performing any better.


root@zgemmah7:~# cat /proc/swaps
Filename Type Size Used Priority
/dev/mmcblk0p7 partition 262140 0 -1
/media/usb/swapfile file 1048572 28 0
root@zgemmah7:~# free
total used free shared buff/cache available
Mem: 1027736 915388 33492 204 78856 79556
Swap: 1310712 28 1310684
root@zgemmah7:~#

using new swapmanager, I have attached the log. The GUI is still showing the same as previously
this has been surprisingly a pain the ****, caused by all the error msgs thrown up by the eMMC partitions.
trying to delete the h7 swap partition (built in the image build) should only disable it...........
So please try this!! .... Again post debug log and results

lincsat
21-06-22, 16:16
OK, that's about the same mate.


16:01:56.6122 [SwapManager][StartSwap] Found a SWAP partition:
16:01:56.6169 [SwapManager][StartSwap] Found a SWAP file on /media/usb/swapfile
16:01:56.6178 [SwapManager] SWAP file is already active on /media/usb/swapfile

I'm guessing the swap partition is the MMC swap, both swaps are active as before.

The swap GUI looks the same, again it won't allow me to deactivate or change what it shows from the manufacturer defined

twol
21-06-22, 18:19
OK, that's about the same mate.


16:01:56.6122 [SwapManager][StartSwap] Found a SWAP partition:
16:01:56.6169 [SwapManager][StartSwap] Found a SWAP file on /media/usb/swapfile
16:01:56.6178 [SwapManager] SWAP file is already active on /media/usb/swapfile

I'm guessing the swap partition is the MMC swap, both swaps are active as before.

The swap GUI looks the same, again it won't allow me to deactivate or change what it shows from the manufacturer defined
No - thats better! Now shows both (just not in display) - I need to be able to display both .. or something.
However I think you will now find only the swap file active

lincsat
21-06-22, 18:26
No - thats better! Now shows both (just not in display) - I need to be able to display both .. or something.
However I think you will now find only the swap file active

Both swaps are active mate, without the mod to make the USB a higher priority, it's still using the MMC first


root@zgemmah7:~# cat /proc/swaps
Filename Type Size Used Priority
/dev/mmcblk0p7 partition 262140 5020 -1
/media/usb/swapfile file 1048572 0 -2

twol
21-06-22, 18:45
Both swaps are active mate, without the mod to make the USB a higher priority, it's still using the MMC first


root@zgemmah7:~# cat /proc/swaps
Filename Type Size Used Priority
/dev/mmcblk0p7 partition 262140 5020 -1
/media/usb/swapfile file 1048572 0 -2

OK need to play with this on my gbue4k which has a swap partition also

twol
22-06-22, 16:38
@Lincsat

So ground rules!
If there is a image built swap partition , it will show as active at startup. Priority = -1 (default)
If you then create (blue button)a swap file (I have allocations up to 1024MB at the moment), it will now show up in the swap file list ----- > then activate (green) button and it will be swapped on as priority 10.
Obviously if you already have setup a swap file, it should appear in the display, and you can deactivate or delete.
Delete will then show the factory swap, if present - which cannot be deleted.

Try it - may crash, if so crash logs please!.

lincsat
22-06-22, 17:44
That appears to be working pretty much as expected. The swap is the cumulative of the machine swap and any generated swap file but as you mentioned, it prioritises the generated swap.