@jasonscottmontoya Check value of menutype column in #__menu table for componentss which disappeared. This column should have value of main

Ok, I've gone in and found the discrepancy - Correcting it per your direction worked. Any idea how this occurred in the first place?

For same reson this query whas not run https://github.com/joomla/joomla-cms/blob/staging/administrator/components/com_admin/sql/updates/mysql/3.7.0-2017-01-17.sql. If you don't use menu named main or menu in frontentd run this query on phpmyadmin. Replace #__ with table prefix.

ok, thanks.

On Mon, May 1, 2017 at 12:24 PM, Wojciech Smoliński <
notifications@github.com> wrote:

For sam reson this query whas not run https://github.com/joomla/
joomla-cms/blob/staging/administrator/components/com_
admin/sql/updates/mysql/3.7.0-2017-01-17.sql. If you don't use menu named
main or menu in frontentd run this query on phpmyadmin.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#15719 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/Aa8_qPqnp3vOU2qZDS5VbG28_laZrOTwks5r1gcxgaJpZM4NNFMa
.

--
Jason Montoya

Path Of The Freelancer Is Now Available On Amazon
https://www.amazon.com/Path-Freelancer-Actionable-Flourish-Freelancing/dp/1540735419/

I had the same issue: my Joomla 3.6.5 menu was named "main" and after upgrade to 3.7 its items went to admin.

@bolando When more or less this website was created?

The site was created a few years ago in Joomla 2.5 and upgraded to Joomla 3.6 a few months ago.

@bolando Have you used a quick start package for a template?

No

Same issue here on multiple - like two dozen - sites (updating from 3.6.5 to 3.7.0). Submitting the named SQL query solved the issue, but where does this issue come from in the first place???

I have also has exactly the same problem - updated from 3.6.5 to 3.7.0. This query addressed the problem successfully.
There is however also another issue since the upgrade - that of the Send Test Email from the Server Tab under Global Configuration - keep getting:
Error
An error has occurred while fetching the JSON data: HTTP 500 status code. Internal Server Error
Wondered if this was perhaps related?

In my case the main menu was being merged into the Components admin menu.

Based on the advice above I gave my main menu a different type string ('top') before attempting the upgrade a second time. This time the menus were not corrupted.

ok, thanks for sharing.

On Wed, May 17, 2017 at 12:19 PM, Clint Buhs notifications@github.com
wrote:

Based on the advice above I gave my main menu a different type string
('top') before attempting the upgrade a second time. This time the menus
were not corrupted.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#15719 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/Aa8_qEhCpBL1LBTH6xGVhbhWRs_34bBLks5r6x4CgaJpZM4NNFMa
.

--
Jason Montoya - 770-265-1933
Path Of The Freelancer Is Now Available On Amazon

http://amzn.to/2p1nDd2 http://amzn.to/2p1nDd2

Thanks for sharing. That sql update snippet brought the admin back all the menu items.

@wojsmol thanks for sharing. worked perfectly. I used a quickstart package though and not upgraded Joomla

@shanenefdt that's an error unrelated to this. Search git please.

Please test PR #16577 as soon as I have updated the testing instructions there. Btw. does anybody know which Joomla version can be used to create a menu type "main" or "menu" for the site (frontend)? Does that work on a 3.6.5? Would be good to know for easy testing instructions.

PR #16577 is ready. Please test.

Guys, please test #16577 (issue tracker link see here), otherwise the issue might not be solved with 3.7.3, and other people will run into the same trouble with menus being messed up when updating a version lower than 3.7.0 to 3.7.3.

Close per fixed in #16577

Set to "closed" on behalf of @franz-wohlkoenig by The JTracker Application at issues.joomla.org/joomla-cms/15719

closed as having PR #16577.

Thanks for Hint, @Quy

Was that solved in 3.7.3? I just upgraded multiple sites (25 in total) from 3.7.2 to 3.7.3 and found out that some entries in the menu where missing. I then check #__menu and I still can see that the menutype for com_banners, com_messages, com_newsfeeds_categories and a lot others are still on "menu" rather then on "main".

The sites I updated where created with different Joomla versions in the past and then upgraded to the latest versions over time (always with uploading the files via FTP and replacing the existing ones, then running the DB upgrade button in the admin GUI).

Running the following:

UPDATE <myDBName>.<myJoomlaDBsuffix>_menu SET menutype = 'main' WHERE menutype = 'menu';

fixed the issue for me

@BastianWie It was already solved in 3.7.0, but that has caused issues for others who had user-defined menus for the frontend. So in 3.7.3 the update statement in the schema update sql for 3.7.0 was corrected. But there was no change made for updating a 3.7.2 to 3.7.3. If in past the update from a version before 3.7.0 to version 3.7.0 has not fixed the problem for some reason, then updating to 3.7.3 will not change anything.

If you say you had a 3.7.2 with menu items of menu type "menu" for the backend (admin), then it means your update to 3.7.0 has not been performed correctly.

The method you use for updating is since a while not officially supported anymore.

The Joomla! update component offers now (beside the live update) the possibility to upload the update package, and so there is really no need anymore to do it in the old (unsupported) way.

See the instructions on how to update in the Joomla! docs.

When you update with the old method, DML (data manipulation language, i.e. insert, update, delete) statements in schema update scripts will not be run, because the db fix only cares about the database schema, i.e. about DDL (data definition language, e.g. create table and so on).

And so then you update in that way, the fix from 3.7.0 for the menu type names is not performed.

When you update a pre-3.7.0 with the Joomla! update component, the complete schema update scripts will be run once if necessary, and so also the fix from 3.7.0 (which was corrected in 3.7.3) will be run, and all will be ok.

A site which has been updated in past from version to version only with the copy or ftp the files and then run the db fix method will very likely be the source of many issues for these reasons (dml of schema updates did not run).

I am not sure if someone (or me) should make a PR here to fix such sites.

Since the Joomla Update Component exists, the copy or ftp and so on method was only an emergency solution in case Joomla Update component cannot be used, it never was the preferable way to update.

So or so it seems to me that the 3.7.0 was not clean somehow.

I think if peopple refuse to follow the updating instructions which are easy to find in the docs and linked in every release announcement, it is not a bug of Joomla, it is a bug of the people. Sorry to say.

Just FYI: I ran into the same issue with a site which was updated over the last few years with with the joomla update component. I also did not have a menu with name "main".

@Hackwar Validate the data in the #__menu table in a copy of the database that was made before the update.

@Hackwar The point is not which method has been used in past. The point is that people still use the "copy or unzip the files and then run db fixer" method to update to 3.7.3 and then complain that the fix from #16577 has not been applied.

Well, one thing is that it was fixed in PR #16577 for the update from pre-3.7.0 versions to 3.7.0, not for updating a 3.7.0 or 3.7.1 or 3.7.2 to 3.7.3.

The reason for that was that I just wanted to fix the wrong schema update script for 3.7.0, assuming that those who already did the update had to fix the problem manually anyway after the update and so have solved their problem, or had to fall back to before the update, which would make the PR #16577 to have an effect (if using the update component) when the do again the update of the pre-3.7.0. to 3.7.3.

And so I assumed that there should not be a version 3.7.0 or 3.7.1 or 3.7.2 having that problem.

If this assumption is wrong, it may make sense to add an update script for 3.7.4 which also includes the fix, so when updating to 3.7.4 using the update component would fix it.

To see if this is necessary it would help if someobody who still has that problem on a 3.7.1 or 3.7.2 or 3.7.3 would provide information here about the content of the "#__menu" and the "#__menu_types" table (columns menutype, title, client_id and in case of "#__menu" also the id should be sufficient).

Guys, do you think I should make a PR which repeats the fix from #16577 in new update script 3.7.4-2017-xx-yy.sql so versions 3.7.0 to 3.7.3 will be fixed. too? I have already prepared that, but am waiting for feedback. But also this would not work if not using the Joomla Update component to do the update to a 3.7.4 (or patched 3.7.4-dev for testing).

We really need to stop using JSchemaChange* as a means for ensuring system integrity. SQL statements that need to be replayable and do not change the database schema need to be moved to a point where they can be triggered in PHP (and inherently reused if needed).

So no. Do not add an update delta to replay that change. If this is such a critical change that it has to be verifyable and something that can be retriggered through that magic black box we call the database fix button (which does more than validate the database if you ever trace its internals), it needs to be handled right and we need to stop monkey patching the system with crap solutions.

@mbabker Then maybe a fix in script.php (which is also called by the database fixer) like it is done already for other stuff could be a way?

I am not sure how critical it is, I never had that issue. But disappearing menus seem critical to me.

In my eyes, the only queries that should be in those changeset files should be queries related to altering the database schema. Queries that are changing data records do not belong there. A database migrations system (for core Joomla that's JSchema*, outside Joomla you have tools like Phinx) is designed for managing schema changes and should only be used for data changes when a schema change necessitates it.

Sur, I already understood that and agree. But now we have it different, and my question was then if it is worth to be done in script.php. The statements of #16577 do not do any harm if multiply applied and the conditions in the statements are not met after the 1st apply.

By the way, couldn´t we use the "Post-Install Messages" to add a news here that the old upgrade way to overwrite files via FTP is no longer supported and shouldn´t be used? At some time it could become critically and I personally think that there are much more persons who never have seen that. The "Post-Install Messages" already informed about the two factor authentication, so why not for the correct way to upgrade a installation? I think informing the users during the next upgrade via "Post-Install Messages" wouldn´t harm and might help to spreed out this important news.

Not much point it being in the post install message - its too late then

http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail
Virus-free.
www.avg.com
http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On 9 July 2017 at 20:52, Bastian W. notifications@github.com wrote:

By the way, couldn´t we use the "Post-Install Messages" to add a news here
that the old upgrade way to overwrite files via FTP is no longer supported
and shouldn´t be used? At some time it could become critically and I
personally think that there are much more persons who never have seen that.
The "Post-Install Messages" already informed about the two factor
authentication, so why not for the correct way to upgrade a installation? I
think informing the users during the next upgrade via "Post-Install
Messages" wouldn´t harm and might help to spreed out this important news.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#15719 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABPH8ctmSeEScUiFO1YVWOJHlflkmKccks5sMS91gaJpZM4NNFMa
.

--
Brian Teeman
Co-founder Joomla! and OpenSourceMatters Inc.
https://brian.teeman.net/ http://brian.teeman.net/

You can still do an upgrade through FTP if you know what you're doing. It's not that FTP upgrades are not supported, but more often than not, users aren't performing all of the required steps to do an upgrade. It was fine in Joomla 1.5 days because the upgrade really way nothing more than "replace these files", but since 1.7 upgrades have supported numerous extra steps under the hood as well (database updates and clearing the system cache being two important tasks).

If you're upgrading outside of using the upgrade component and you basically aren't following the steps outlined in https://gist.github.com/mbabker/d7bfb4e1e2fbc6b7815a733607f89281 (not saying you have to use that script but it does trigger all the things), your upgrade can't be considered complete.

@mbabker Does this script also execute UPDATE queries from sql files? I remember cases from the Polish Joomla forum where most probably did not do it

My script uses the update component's model to emulate everything that happens after a normal update has unpacked the update package's contents onto your site. That code calls into the JSchema API, and that is the API that will not run data statements (INSERT, UPDATE, etc.) after the first time a file is parsed.

So if the data queries are somewhere in our admin script.php file then they can always be checked and executed in that magic black box fix routine (and inherently anything that uses it). If the data queries are only in SQL files, they can only be processed once and cannot be validated again.

Would it be possible to rewrite this script to emulate upload and install. Omitting the zip file download phase? It should temporarily terminate https://docs.joomla.org/J3.x:Update_from_3.7.x_to_3.7.1_Timeout and any problem, updating to 3.8.x, 3.9.0 and 4.0 resulting from the same entity.

It basically does emulate upload and install. The difference is you manually unpack the ZIP archive on your server then trigger the script whereas upload and install unpacks the package the same way the "normal" update routine does.

Suffering badly from this issue.

I have several joomla sites all 100% up to date i.e using 3.7.3 (I didn't notice missing anything during updates as those components are rarely used).

I've tried all fixes in:
https://issues.joomla.org/tracker/joomla-cms/15719

and:
#16852

but I always run into an error like #1054 - Unknown column 'oldmain' in 'where clause' or #1054 - Unknown column 'client_id' in 'where clause'

I've reinstalled via joomla update and ftp but nothing works.

When I look at the Menus > Menu Items > Administrator I see the missing items all set to Menu but the Select Menu is empty.

In the databasse I can see all items again in #__menu (see: http://i.share.pho.to/9d28ee92_o.png), and when I look in the #_menu_types I only see front end menus (see:http://i.share.pho.to/9d28ee92_o.png), should I see one for Menu or a column for asset_id?

Any help would be greatly appreciated, going out of my mind here!

@janeblonde

Unknown column 'client_id' in 'where clause'

Run this query in phpmyadmin

ALTER TABLE `#__menu_types` ADD COLUMN `client_id` int(11) NOT NULL DEFAULT 0;

Replace #__ with dbprefix

@wojsmol
Thank you, I've ran that and then also the below query which for the first time has been successful however there is no change in the admin menu, I've also rebuild the menu and cleared caches, but still no change. So some progress but still no cigar:(

Here's how the tables now look:
http://i.share.pho.to/66c367ce_o.png
http://i.share.pho.to/9c154962_o.png

UPDATE `e8w6u_menu`
   SET `menutype` = 'main_is_reserved_133C585'
 WHERE `client_id` = 0
   AND `menutype` = 'main'
   AND (SELECT COUNT(`id`) FROM `e8w6u_menu_types` WHERE `client_id` = 0 AND `menutype` = 'main') > 0;

UPDATE `e8w6u_modules`
   SET `params` = REPLACE(`params`,'"menutype":"main"','"menutype":"main_is_reserved_133C585"')
 WHERE `client_id` = 0
   AND (SELECT COUNT(`id`) FROM `e8w6u_menu_types` WHERE `client_id` = 0 AND `menutype` = 'main') > 0;

UPDATE `e8w6u_menu_types`
   SET `menutype` = 'main_is_reserved_133C585'
 WHERE `client_id` = 0 
   AND `menutype` = 'main';

UPDATE `e8w6u_menu`
   SET `client_id` = 1
 WHERE `menutype` = 'main';

UPDATE `e8w6u_menu`
   SET `menutype` = 'main'
 WHERE `client_id` = 1 
   AND `menutype` = 'menu';

UPDATE `e8w6u_menu`
   SET `menutype` = 'main',
       `client_id` = 1
 WHERE `menutype` = 'menu'
   AND (SELECT COUNT(`id`) FROM `e8w6u_menu_types` WHERE `client_id` = 0 AND `menutype` = 'menu') = 0;

DELETE FROM `e8w6u_menu_types`
 WHERE `client_id` = 1
AND `menutype` IN ('main', 'menu');

@janeblonde You use custom admin menu named Admin Menu?

@wojsmol
I have a front end menu called adminmenu (title:Admin Menu), this contains some links for front end editing, do you think this could be the issue?

@janeblonde No, I needed more information about this menu.See #15719 (comment)

@wojsmol
I don't have a custom admin menu, it's the system default for admin, it looks like this:
http://i.share.pho.to/f563e442_o.png
Note: if you click Select Menu there is nothing to select. If I create a custom admin menu then this list disappears.

@janeblonde ok, see #15719 (comment)

@wojsmol
Do you think it could be a missing menu?
All missing menu items in #__menu have main (see:http://i.share.pho.to/b9c3ceb6_o.png)
but in the #__menu_type table there is no corresponding menu for main (nor menu).

@janeblonde See this comment https://github.com/joomla/joomla-cms/blob/2d341103cdd6b6fe652c17776e63fb9f6a17dd7d/administrator/components/com_admin/sql/updates/mysql/3.7.0-2017-01-17.sql#L52-L55 So there is something else is wrong in #__menu table, see this issue #15938 - if nested tree is broken then menu mey disapered.

@wojsmol
I've ran that query, I don't see that it's made a change (see: http://i.share.pho.to/8f47a304_o.png & http://i.share.pho.to/34fcc001_o.png). There is no change on the admin, they all look to be set to Menu and Published (see: http://i.share.pho.to/aefd50a6_o.png).

re #15938 I don't see an error like:

Failed publishing 1 menu item as at least one of its parents is unpublished or one of its children is checked out.

But what is interesting is the mention of Banners, this is not in my list of published Admin menu items and it is included as a menu item (see:http://i.share.pho.to/6db3f6a2_o.png) but if I compare them in the #__menu table they seem more or less the same.

systeminfo-2017-07-25T13_50_00+00_00.txt for reference I've added the system info file here, the highlights would be:
Linux cloud.jai.ie 3.10.0-229.14.1.el7.centos.plus.x86_64
Database Version | 5.5.52-MariaDB
PHP Version | 7.0.21

@janeblonde Plese attach sql dump of #__menu if possible.

certainly...
it contains #__menu_types and #__menu.
dev2.zip

@mbabker Plese try download attached zip - I cen't - for me this is github issue whitch shoud be reported.
@janeblonde Plese upload this zip to your server temporally.

@wojsmol
here you go
http://jai.ie/dev2.zip